Palantir began investigating solutions, but options like Microsoft Office 365 or CKEditor 5 didn’t satisfy them. Using Microsoft 365 meant the team couldn’t bring the entire workflow into Drupal, limiting collaborative editing functionality, and CKEditor 5’s client-server model was a security no-go because it sent their data into an unapproved cloud environment. Stuck between not having a full Drupal workflow and this security limitation, Palantir reached for an alternative solution, creating the EditTogether tool. This process took over a year and was powered by a dedicated team of Palantir developers, technical architects, strategists, a designer, a product owner, and a team lead.
A configurable plug-in system via the Drupal Text Editor API
Palantir’s team tackled the client’s challenges one by one. They started off by attaching EditTogether’s rich text editor to Drupal via the Drupal Text Editor API. They did this because Drupal’s Text Editor API provides a plug-in system to not only add new text editors, but also a config entity for configuring them. “That config entity is what eventually becomes the text formats that you see in Drupal, like your Basic HTML, Restricted HTML, Full HTML, etc.,” says Palantir’s Senior Engineer, Alex Jones.
Flexible rich text editors built using ProseMirror
Next, the team turned to ProseMirror, a flexible toolkit for building rich text editors. ProseMirror enabled the team to instantiate configurable text editors and configure them for specific contexts and content types. “This way you can have text editors with very limited options or a plethora of options, and you can configure them dynamically,” says Alex.
Syncing data with Yjs
To get the collaboration side of things working with ProseMirror, the team used Yjs, a shared editing framework in JavaScript that syncs common data structures. “It can sync hashmaps, arrays, and XML content, but it'll do it in real-time, collaboratively with other people,” says Alex.
Form-level collaboration from ProseMirror and Yjs combined
Hooking Yjs up to ProseMirror and syncing ProseMirror’s state between multiple users gave the team the ability to see cursor positions with fellow content editors and actually edit in real-time. Connecting the two also provides a Yjs document that allows users to insert multiple fields into a single collaborative session. “Theoretically, you can have an unlimited amount of fields tracked collaboratively in your document at one time,” says Alex. By providing flexible editor instances, a data structure for managing content, and an ability to sync data structures in real-time, ProseMirror and Yjs together led to the end goal: form-element-level collaboration. “Using them together allows us to extend collaboration from beyond just the body field to the form element level and actually have collaboration across every form element,” says Alex.
Reliable, secure, and affordable real-time connection via WebRTC
When it came time to actually connect users in real-time, the team had some barriers to overcome. The team looked into two communication model options for RT collaboration, client-server or peer-to-peer.
The team explored WebRTC’s reliability as a peer-to-peer model. WebRTC’s sole purpose is to connect clients with other clients, making it highly dependable for video calls, message services, and general data transfer, a point in its favor.
Security considerations also had to be taken into account during development, since, as with many government agencies, our client had strict limitations on where content can be hosted. The team again leaned toward WebRTC, since the site’s content state is only exposed to collaborators. “Your content never leaves the stewardship of the people who can already access your site,” says Alex. “For security reasons, that's much more preferable for us than WebSocket because there isn't a third party that can view your content.”
Finally, hosting through WebRTC means significantly lower costs than via client-server models. “All of the intensive CPU power that's required for the WebSocket model is actually deferred to each client machine when you're editing via WebRTC. Because the client machines are actually responsible for managing the state of the document, WebRTC is orders of magnitude cheaper than WebSocket,” says Alex.
Granular control and real-time collaboration
After over 13 months of work, The Palantir team delivered a real-time collaboration framework for Drupal 10 and 11 with many exciting improvements to the old system.
- Real-time collaboration in content editing. This functionality allowed for the creation of workflows and processes based on the most recent updates, while also eliminating the need to develop content outside Drupal.
- New plug-in system. The plug-ins enabled granular control over all aspects of EditTogether. Developers could extend EditTogether using custom modules, with extensions including toolbar buttons and functionality, ProseMirror plug-ins (cursors, history, etc.), and major features like collaboration at the form level with commenting inline suggestions and change tracking.
- Auto-saving and Yjs document revisions. Auto-saving in real-time is based on a timeout that the user sets. The Yjs documents are then stored in fields attached to each collaborative entity. Auto-saving occurs independently from saving a node and is continually updated against a forward revision until the node is saved. The Yjs documents provide a granular change history between Drupal revisions, including what the change was, when it happened, and who changed it.
- Real-time collaboration at the form element level. Binding Yjs to form elements using Javascript behaviors offered control over collaboration at the element level, allowed core and custom fields to use collaboration, and enabled collaboration to be expanded to any form. “When we target that form element level, it means anywhere that the form element shows up in Drupal, we can sync it collaboratively and in real-time,” explains Alex.
- WebRTC infrastructure and hosting options. EditTogether can be connected to a variety of different managed services using the plug-in system, including the default signaling server (packaged with Yjs) and Azure Web PubSub.
- Increased data sovereignty. WebRTC ensures that content is only shared between collaborators, not third parties, offering collaboration for security-conscious organizations where client-server models aren’t an option.
- A modular and extensible Rich Text Editor. Media library and Entity browser support are embedded in the text editor itself.
- Easy integrations. The new system was primed to integrate with Drupal content moderation workflows and user access controls.
- Basic WYSIWYG. A variety of options include Bold, Italic, Strikethrough, Ordered List, Unordered List, Indent, Outdent, Superscript, Subscript, Link, Blockquote, Code, Tables, Heading selection, Undo, Redo, Remove formatting, Align Left, Align Right, and Align Center.
