Pushing Ragna UI to its limits #239
Replies: 3 comments 19 replies
-
|
Thanks for providing these insights! I believe large numbers of ChatMessages can be addressed by adding pagination. I created an issue here: holoviz/panel#6021 I'll also see whether ChatMessage can be optimized to render a bit quicker. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a ton for this in-depth post @Tengal-Teemo! I'll convert the actionable items here into into issues for Ragna and Panel.
Apart from the actual SVG loading, we are using a ragna/ragna/deploy/_ui/central_view.py Line 632 in 7b17652 I don't think there is an option to automatically hide items based on width. #224 just limited the maximum number we are going to create, but ultimately this should be dynamic.
I've advocated for caching from the beginning, but we left it out for simplicity in the first release. So I agree this is needed. If we assume that each user only has at most one session, implementing caching should be fairly straight forward. |
Beta Was this translation helpful? Give feedback.
-
|
Param 2.0.2, once released, should provide another 10% speedup. |
Beta Was this translation helpful? Give feedback.
-
|
Oh yes I should mention I believe that Ragna required version 1.3.4. If it's been fixed in the new version that should fix a lot of things. @pmeier |
Beta Was this translation helpful? Give feedback.
-
|
Ragna never required 1.3.4 Any 1.3.* will do. 1.3.4 was just the latest release when you were testing. |
Beta Was this translation helpful? Give feedback.
-
|
Apologies, thank you for clearing that up |
Beta Was this translation helpful? Give feedback.
-
|
No worries. For future references, here is the specification: Line 31 in 0bd014a Note that we just bumped the minimum required version in #259 for the |
Beta Was this translation helpful? Give feedback.
-
|
@Tengal-Teemo Any chance you could share the code you used to profile and generate the timings? |
Beta Was this translation helpful? Give feedback.
-
|
@philippjfr Sure. For panel, I modified the from timeit import default_timer as timer()
def _update_model(
self, events: Dict[str, param.parameterized.Event], msg: Dict[str, Any],
root: Model, model: Model, doc: Document, comm: Optional[Comm]
) -> None:
start = timer()
msg = dict(msg)
inverse = {v: k for k, v in self._property_mapping.items() if v is not None}
preprocess = any(inverse.get(k, k) in self._preprocess_params for k in msg)
# ALERT: Find a better way to handle this
if 'styles' in msg and root is model and 'overflow-x' in msg['styles']:
del msg['styles']['overflow-x']
obj_key = self._property_mapping['objects']
update_children = obj_key in msg
if update_children:
old = events['objects'].old
children, old_children = self._get_objects(model, old, doc, root, comm)
msg[obj_key] = children
msg.update(self._compute_sizing_mode(
children,
dict(
sizing_mode=msg.get('sizing_mode', model.sizing_mode),
styles=msg.get('styles', model.styles),
width=msg.get('width', model.width),
min_width=msg.get('min_width', model.min_width),
margin=msg.get('margin', model.margin)
)
))
else:
old_children = None
with hold(doc):
update = Panel._batch_update
Panel._batch_update = True
try:
with freeze_doc(doc, model, msg, force=update_children):
super()._update_model(events, msg, root, model, doc, comm)
if update:
stop = timer()
print("Updated Model (Returning): " + str(stop - start))
return
from ..io import state
ref = root.ref['id']
if ref in state._views and preprocess:
state._views[ref][0]._preprocess(root, self, old_children)
finally:
Panel._batch_update = update
stop = timer()
print("Updated Model: " + str(stop-start))This is all I did, further information on how to create the cases in which the timing issues persisted can be found in the original post. Let me know if you have more questions. Edit: clarification: I just added the lines using timer() |
Beta Was this translation helpful? Give feedback.
-
|
For constructing the chat messages I just did this inside the constructor for RagnaChatMessage start = timer()
super().__init__(**(params | kwargs))
print("Constructed ChatMessage in "+str(timer() - start))On line 146 of |
Beta Was this translation helpful? Give feedback.
-
Pushing Ragna UI to its limits
I thought it would be a good idea to try limit testing Ragna within what would be considered its reasonable use-case. I have tested some limits that are significant to my work, and come up with some interesting findings and potential solutions.
System:
For these investigations, I added some rudimentary logging to both Ragna and Panel using the python library timeit.
Large numbers of documents
The only issue with this has already been addressed in #224, but I'd just like to quantify the performance improvement. When loading a corpus of 2224 documents, with the pills header the upload took roughly 221.54 seconds.
Patching out the pills header and performing the same upload yielded 0.15 seconds, quite a large improvement.
The primary cause of this appears to be the way that panel implements loading. The 64 bit encoded SVG is uploaded for every single element on that page, including all pills and all chat messages. This was observed with tcpflow producing the following logs from the command
sudo tcpflow -p -c -i any port 31477 | grep "pn-dots"This is not the full message. The string
PHN2ZyB4bWxucz0iaHR0cDovLis the beginning of the SVG for the loading icon. This is sent for every single element on the page as observed through TCP flow. This is more of a bug in panel than ragna, and unless the page is composed of thousands of elements poses no issue.Note: Quite strange is that the long setting of the chat only appeared after the first upload of the corpus. When uploaded for the first time, the loading time was noticeably shorter (only 25 seconds) and the logs show the current chat set in only 5.6 seconds.
For the remaining tests and experiments, I patched out the pills by commenting out lines 596-623 in ragna/_ui/central_view.py
Large numbers of chat messages
When switching between chats with a large number of messages, there is a very large delay. The main cause is determined to be from 2 places. The first is the re-construction of the list of RagnaChatMessages. The second is the re-rendering of the bokeh model after the update. See below for logs from swapping between 2 chats (100 Questions, 100 Responses, 2 character query, 2700 character response)
Reconstruction of the list of RagnaChatMessages
This takes a long time because ragna is built from panel. Analysing this further, almost the entire time constructing the RagnaChatMessage comes from constructing the underlying panel.chat.ChatMessage object underneath. The best solution for this is to cache the RagnaChatMessage and ChatMessage objects when entering and exiting chats, or upon generation. With this implemented, performance is improved:
The remaining issue is the update time. This also appears to be an underlying issue with panel, as when the chat interface is recreated the bokeh model needs to be re-rendered with a massive number of changes. This was verified by recreating a base case using panel and a similar number of chat messages. This was verified by porting RagnaChatMessages to a separate panel app and testing them against the most basic panel.chat.ChatMessage objects.
In my opinion the best way to fix this would be to implement central_view inside of panel tabs, and tie each tab to a chat id. This means that the bokeh model is never re-rendered with a lot of objects at once (provided the tabs aren't set to dynamic).
Large numbers of chats
There appears to be a bug with having 25 or more chats. You cannot have more than 25 chats. Upon creation of the 25th chat, the modal will say "an error occurred". Clicking "Start Conversation" again will result in the creation of the chat, but any more presses of the + New Chat button will result in the following error message:
I will be creating an issue describing this bug properly. Edit: this issue has been created here #240
Overall Consensus
I think that the method of repeatedly reconstructing the chats needs to be removed. At least, some kind of caching of the chat_message objects should be done on the server side so they don't have to be recreated every time you change chats. Ultimately even with all the changes and patches suggested here, it would still take a very long time to load all chats on refresh, speaking to the issues with the underlying panel library. Perhaps a more performant library could be used, but I have not done any research as to what or making any kind of comparison. I'd be happy to help with any changes that are made as an outcome of this post as well. Thanks for reading this long discussion post 😄
Beta Was this translation helpful? Give feedback.
All reactions