Web Editor Support

Since version 2.9 Xtext offers an interface for integration of text editors in web applications. The text editors are implemented in JavaScript, and the language-related services such as code completion are realized through HTTP requests to a server-side component.

The Client

Xtext supports three JavaScript text editor libraries:

• Orion
• Ace
• CodeMirror

The set of supported language-related services depends on which editor library is chosen, as shown in the following table.

	Orion	Ace	CodeMirror
Content assist
Validation
Syntax highlighting
Semantic highlighting
Mark occurrences
Hover information
Formatting
Code generator
Persistence

The JavaScript integration of Xtext uses RequireJS for managing modules and their dependencies. The main module is xtext/xtext-orion, xtext/xtext-ace, or xtext/xtext-codemirror, depending on which editor library is used. Examples for loading these libraries together with Xtext can be generated by the WebIntegrationFragment by setting the option generateHtmlExample = true (see The Language Generator). A minimal setup based on WebJars is shown in the following listings.

Orion

require.config({
    paths: {
        "text": "webjars/requirejs-text/<version>/text",
        "jquery": "webjars/jquery/<version>/jquery.min",
        "xtext/xtext-orion": "xtext/<version>/xtext-orion"
    }
});
require(["orion/code_edit/built-codeEdit-amd"], function() {
    require(["xtext/xtext-orion"], function(xtext) {
        xtext.createEditor();
    });
});

Since Orion is not available on WebJars, it has to be downloaded manually at download.eclipse.org.

Ace

require.config({
    paths: {
        "jquery": "webjars/jquery/<version>/jquery.min",
        "ace/ext/language_tools": "webjars/ace/<version>/src/ext-language_tools",
        "xtext/xtext-ace": "xtext/<version>/xtext-ace"
    }
});
require(["webjars/ace/<version>/src/ace"], function() {
    require(["xtext/xtext-ace"], function(xtext) {
        xtext.createEditor();
    });
});

CodeMirror

require.config({
    paths: {
        "jquery": "webjars/jquery/<version>/jquery.min",
        "xtext/xtext-codemirror": "xtext/<version>/xtext-codemirror"
    },
    packages: [{
        name: "codemirror",
        location: "webjars/codemirror/<version>",
        main: "lib/codemirror"
    }]
});
require(["xtext/xtext-codemirror"], function(xtext) {
    xtext.createEditor();
});

JavaScript API

xtext.createEditor(options)

Creates one or more editor instances and initializes Xtext services for them. The available options are described below. The return value is the created editor if exactly one was created, or an array of editors otherwise:

var editor = xtext.createEditor();

For Orion the editor is not returned directly, but through a promise:

xtext.createEditor().done(function(editorViewer) {
    ...
});

If the parent element for the editor is not specified through the options, this function looks for an element with id="xtext-editor". If none exists, the function looks for elements with class="xtext-editor".

xtext.createServices(editor, options)

Initializes Xtext services for an already created editor. Use this variant if you want full control over the editor creation.

xtext.removeServices(editor)

Removes all services and associated listeners from the given editor. The JavaScript-based syntax highlighting is not affected by this operation.

Options

Options can be specified with an object passed to the createEditor or createServices functions:

xtext.createEditor({
    resourceId: "example.statemachine",
    syntaxDefinition: "xtext/ace-mode-statemachine"
});

Alternatively, if createEditor is used, options can be set as attributes of the HTML element into which the editor is created. The attribute name is derived from the JavaScript option name by converting CamelCase to hyphen-separated and prefixing with data-editor-, e.g. resourceId becomes data-editor-resource-id:

<div id="xtext-editor"
    data-editor-resource-id="example.statemachine"
    data-editor-syntax-definition="xtext/ace-mode-statemachine"></div>

See below for a full list of available options.

Getting the Text Content

The text content is either loaded from the Xtext server or provided through JavaScript. In either case, the Xtext server needs to identify the language you are using in order to process any requests. A language is usually identified using a file extension or a content type. The file extension can be specified with the xtextLang option, while the content type is given with the contentType option.

Loading text from the server

If you specify a value for the resourceId option, the language may be identified from the file extension included in that id. In this case it is not necessary to provide the xtextLang option. Another effect of using resourceId is that the server will try to load the respective resource from its persistence layer. The Xtext server does not define any default persistence, so you have to customize it as described in the Persistence section in order to use this approach.

The resourceId string may contain whatever information you need to identify a specific text content on the server. Usually the URI syntax is used, but this is not mandatory.

Providing text through JavaScript

The persistence layer of the Xtext server is totally optional, so you can use whatever means you like to fetch the content, e.g. generate it in the client or request it through your own service. If the DOM element into which the editor is created already contains some text, this text is used as the initial content. Otherwise the editor is initially empty, and you can use the API of the chosen editor framework to change the content whenever you like.

If you would like to specify a resourceId without using the persistence services shipped with Xtext, you can do so by setting the option loadFromServer to false.

Operation Modes

The web integration of Xtext supports two operation modes, which are described in the following.

Stateful mode

This mode is active when sendFullText is false, which is the default setting. In stateful mode, an update request is sent to the server whenever the text content of the editor changes. With this approach a copy of the text is kept in the session state of the server, and many Xtext-related services such as AST parsing and validation are cached together with that copy. This means that services run much faster on the server, and the message size of most requests is very small. Use this mode in order to optimize response times of service requests.

Stateless mode

This mode is active when sendFullText is set to true. In this case no update request is necessary when the text content changes, but the full text content is attached to all other service requests. This means that the text has to be parsed again for each request, and the message size is proportional to the size of the text content. Use this mode only when you want to avoid server-side sessions and respective session ids stored in cookies.

Syntax Highlighting

In contrast to semantic highlighting, syntax highlighting can be computed purely in JavaScript without requiring a connection to the server. This is realized with the highlighting capabilities of the employed editor libraries. All three libraries offer a JavaScript API for specifying highlighting rules. The WebIntegrationFragment of the Xtext language generator is able to generate a highlighting specification that includes keywords and some basic tokens such as numbers and comments (see The Language Generator). If the highlighting generated in this way is not sufficient, you can still implement the specification yourself following the documentation of the chosen editor library.

The path to the highlighting specification is set with the syntaxDefinition option while setting up the Xtext editor. If no value is specified for that option, it is built from the xtextLang option in the form 'xtext-resources/{xtextLang}-syntax' (Orion) or 'xtext-resources/mode-{xtextLang}' (Ace). For CodeMirror syntax highlighting is configured by registering a mode and setting the mode option accordingly (default mode name: 'xtext/{xtextLang}').

Invoking Services

The createEditor and createServices functions set up listeners for editor events such that most services are invoked automatically while using the editor. However, all services can also be invoked programmatically using the xtextServices object that is attached to each editor instance. For example, the following code saves the resource associated with the editor when the button with the id save-button is clicked:

var editor = xtext.createEditor();
$("#save-button").click(function() {
    editor.xtextServices.saveResource();
});

The following functions are available, provided that the respective services are enabled and supported by the employed editor library. All functions return a promise that is eventually resolved to the requested data. Furthermore, all these functions can be supplied with an options parameter to override some of the options declared when the editor was built.

• getContentAssist()
  Returns the content assist proposals in the format explained in the Content Assist section.
• validate()
  Returns the validation result. The returned object has a property issues, which is an array of objects with the following properties:
  • description: A description to be displayed to the user.
  • severity: one of 'error', 'warning', or 'info'
  • line: the line where the issue occurs (one-based)
  • column: the column where the issue occurs (one-based)
  • offset: the character offset in the document (zero-based)
  • length: the length of the affected text region
• computeHighlighting()
  Returns the text styling data for semantic highlighting (see the Semantic Highlighting section).
• getOccurrences()
  Returns the occurrences of the element at the current cursor position (see the Mark Occurrences section).
• getHoverInfo()
  Returns the hover information at the current cursor position (see the Hover Information section).
• format()
  Formats the current selection (or the whole document if the selection has zero length) and returns the formatted text.
• generate()
  Returns the document generated from the associated resource. If more than one document has been generated, an object with a property documents is returned, which is an array of objects with the properties name, contentType, and content.
• loadResource()
  Loads the associated resource and returns an object with properties fullText and dirty. If the resource has been modified during the current session, the modified version is returned even if the page is reloaded.
• saveResource()
  Saves the current state of the resource.
• revertResource()
  Reverts the resource to the last saved state and returns an object with properties fullText and dirty.
• update()
  Computes the difference between the current editor content and the last version that has been committed to the server. If there is a difference, an update request is sent to refresh the server state. The return value has a single property stateId, which is an identifier for the new server state. All requests have to include the last obtained state identifier in order to succeed.

Full List of Options

• baseUrl
  The path segment where the Xtext service is found (default: '/'). See the serviceUrl option.
• contentAssistCharTriggers
  Characters that invoke the content assist service when typed (Orion only).
• contentAssistExcludedStyles
  Excluded styles for character triggers (Orion only).
• contentType
  The content type included in requests to the Xtext server. If no content type is given, the file extension in the resourceId option is used to determine the language.
• dirtyElement
  An element into which the dirty status class is written when the editor is marked dirty; it can be either a DOM element or an id for a DOM element.
• dirtyStatusClass
  A CSS class name written into the dirtyElement when the editor is marked dirty (default: 'dirty').
• document
  The document; if not specified, the global document is used.
• enableContentAssistService
  Whether content assist should be enabled (default: true).
• enableCors
  Whether CORS should be enabled for service request (default: false).
• enableFormattingAction
  Whether the formatting action should be bound to the standard keystroke Ctrl+⇧+F (Ctrl+⇧+S on CodeMirror) / ⌘+⇧+F (default: false).
• enableFormattingService
  Whether text formatting should be enabled (default: true).
• enableGeneratorService
  Whether code generation should be enabled (default: true). No default keystroke is bound for the generator, so it must be triggered through JavaScript code.
• enableHoverService
  Whether mouse hover information should be enabled (default: true).
• enableHighlightingService
  Whether semantic highlighting should be enabled (default: true). In contrast to syntax highlighting, semantic highlighting is computed on the server.
• enableOccurrencesService
  Whether marking occurrences should be enabled (default: true).
• enableSaveAction
  Whether the save action should be bound to the standard keystroke Ctrl+S / ⌘+S (default: false).
• enableValidationService
  Whether validation should be enabled (default: true).
• loadFromServer
  Whether to load the editor content from the server. The default value is true if the resourceId option is set and false otherwise.
• mouseHoverDelay
  The number of milliseconds to wait after a mouse hover before the information tooltip is displayed (default: 500).
• parent
  The parent element into which the editor should be created. It can be either a DOM element or an id for a DOM element (default: 'xtext-editor').
• parentClass
  If the parent option is not given, this option is used to find elements that match the given class name (default: 'xtext-editor').
• resourceId
  The identifier of the resource displayed in the text editor. This option is sent to the server to communicate required information on the respective resource.
• selectionUpdateDelay
  The number of milliseconds to wait after a selection change before Xtext services are invoked (default: 550).
• sendFullText
  Whether the full text shall be sent to the server with each request (default: false). Use this if you want the server to run in stateless mode. If the option is inactive, the server state is updated regularly.
• serviceUrl
  The URL of the Xtext servlet. If no value is given, it is constructed using the baseUrl option in the form {location.protocol}//{location.host}{baseUrl}xtext-service.
• showErrorDialogs
  Whether errors should be displayed in popup dialogs (default: false).
• syntaxDefinition
  A path to a JS file containing a syntax definition. Set this option to 'none' to disable syntax highlighting. See the Syntax Highlighting section for more details.
• textUpdateDelay
  The number of milliseconds to wait after a text change before Xtext services are invoked (default: 500).
• xtextLang
  The language name (usually the file extension configured for the language). This is used to set the resourceId and syntaxDefinition options if they have not been specified.

The Server

The language-specific resources are provided through HTTP requests which are processed by the XtextServlet. This class is also responsible for managing the lifecycle of the language Injector (see Dependency Injection). The default approach is to create an injector when the servlet is started and to register it in the IResourceServiceProvider.Registry. In order to override the default behavior, you can change or add bindings in the <LanguageName>WebModule or <LanguageName>IdeModule of your language.

The usual way to include the Xtext servlet in a server application is to create a subclass of XtextServlet, override init() and destroy() to manage the runtime resources, and add a WebServlet annotation with urlPatterns = "/xtext-service/*" as parameter. See MyXtextServlet for an example.

If you want to implement your own communication channel to provide the language-specific services without using XtextServlet, you can do so by injecting an instance of XtextServiceDispatcher and calling getService(IServiceContext). The input to this method is an IServiceContext, which must be implemented to provide the request parameters and the client session. The return value is a descriptor that can be used to invoke the actual service. Furthermore, XtextServiceDispatcher can be subclassed in order to add custom services with access to the document AST and all related Xtext APIs.

The following sections describe how to customize the standard services for web editors.

Content Assist

Content assist proposals for cross-references are created by IdeCrossrefProposalProvider, while for other grammar elements IdeContentProposalProvider is used. In order to customize the proposals, create subclasses of these providers and register them in your IDE Guice module.

IdeContentProposalProvider has one _createProposals(...) dispatch method for each type of grammar element. In most cases the best choice is to override the method for Assignments and to filter the correct assignments by comparing them with the instances in the generated GrammarAccess of your language. A proposal is submitted by creating and configuring an instance of ContentAssistEntry and passing it to the given acceptor. This entry class contains all information required to display the proposal in the web browser and to apply it to the document. Usually it is sent to the client in JSON format.

The typical customization point for IdeCrossrefProposalProvider is the method createProposal(...), which is called for each element found by the scope provider. Here you can fine-tune the information to put into the ContentAssistEntry.

Semantic Highlighting

The default behavior of Xtext editors is to have no semantic highlighting (syntactic highlighting, e.g. for keywords and strings, is done on the client side as described in Syntax Highlighting). In order to add styling to your text, create a subclass of DefaultSemanticHighlightingCalculator and override highlightElement(...). Here you can mark text regions with CSS classes by submitting a text offset, a length, and the CSS class name to the given acceptor. You can specify the actual text style in a CSS file that is included by the web page containing the Xtext editor.

Mark Occurrences

The service for marking occurrences of a selected element is handled by OccurrencesService. Here you can override filter(EObject) to exclude some elements from this service. The actual text regions to be marked are computed automatically from the cross-references present in the model.

Hover Information

The information displayed for mouse hover popups is created in HTML format and consists of two parts: a title and a description.

The title is composed of a text label and an image. The label is computed by the INameLabelProvider, and the default is the value of the name property of an element, if it exists. The image is determined by an implementation of IImageDescriptionProvider. The default behavior is to generate a <div> and annotate it with a CSS class of the form <class>-icon, where <class> is the name of the EClass of the corresponding element. The actual images can be assigned to these classes in a CSS file using the background-image CSS property.

The description part of the hover popup is determined by the IEObjectDocumentationProvider. For this part the default content is fetched from the comments in the document: if the definition of the element is preceded by a multi-line comment (e.g. /* ... */), the content of that comment is used as description.

Persistence

Without further adaptation the Xtext server does not provide any persistence functionality. As described in the Getting the Text Content section, there are multiple ways to fill the web editor with text. If you would like to use the persistence service included in the Xtext server, you need to implement IServerResourceHandler. The get and put operations declared in that interface should delegate to the persistence layer you would like to connect. For a simple example see FileResourceHandler.

---

Next Chapter: LSP Support