diff options
Diffstat (limited to 'editor/docs')
| -rw-r--r-- | editor/docs/ChangJie.png | bin | 0 -> 544 bytes | |||
| -rw-r--r-- | editor/docs/EditorModuleSpecificRules.rst | 215 | ||||
| -rw-r--r-- | editor/docs/EditorModuleStructure.rst | 219 | ||||
| -rw-r--r-- | editor/docs/IMEHandlingGuide.rst | 1091 | ||||
| -rw-r--r-- | editor/docs/candidatewindow.png | bin | 0 -> 5667 bytes | |||
| -rw-r--r-- | editor/docs/converted_composition_string.png | bin | 0 -> 1260 bytes | |||
| -rw-r--r-- | editor/docs/index.rst | 12 | ||||
| -rw-r--r-- | editor/docs/inputting_composition_string.png | bin | 0 -> 1303 bytes | |||
| -rw-r--r-- | editor/docs/raw_composition_string.png | bin | 0 -> 1302 bytes |
9 files changed, 1537 insertions, 0 deletions
diff --git a/editor/docs/ChangJie.png b/editor/docs/ChangJie.png Binary files differnew file mode 100644 index 0000000000..04d872e258 --- /dev/null +++ b/editor/docs/ChangJie.png diff --git a/editor/docs/EditorModuleSpecificRules.rst b/editor/docs/EditorModuleSpecificRules.rst new file mode 100644 index 0000000000..dbe4a2b12f --- /dev/null +++ b/editor/docs/EditorModuleSpecificRules.rst @@ -0,0 +1,215 @@ +############################ +Editor module specific rules +############################ + +The editor module has not been maintained aggressively about a decade. Therefore, this module needs +to be treated as a young module or in a transition period to align the behavior to the other +browsers and take modern C++ style. + +Undoubtedly, this editor module is under rewritten for modern and optimized for current specs. +Additionally, this module does really complicated things which may cause security issues. +Therefore, there are specific rules: + +Treat other browsers behavior as standard if the behavior is reasonable +======================================================================= + +The editing behavior is not standardized since as you see too many lines in the editor classes, the +number of cases which need to handle edge cases is crazy and that makes it impossible to standardize. +Additionally, our editor behavior is not so stable. Some behaviors were aligned to Internet Explorer, +some other behaviors were not for making "better" UX for users of email composer and HTML composer +which were in SeaMonkey, and the other browser engines (Blink and WebKit) have same roots but the +behavior is different from both IE and Gecko. + +Therefore, there were no reference behavior. + +In these days, compatibility between browsers becomes more important, and fortunately, the behavior +of Blink (Chrome/Chromium) which has the biggest market share is more reasonable than ours in a lot +of cases. Therefore, if we get web-compat issue reports, we should align the behavior to Blink in +theory. + +However, if Blink's behavior is also odd, this is the worst case. In this case, we should try to +align the behavior to WebKit if and only if WebKit's behavior is different from Blink and +reasonable, or doing something "better" for hiding the issue from web-apps and file an issue to the +Editing Working Group with creating a "tentative" web-platform test. + +Don't make methods of editor classes public if they are used only by helper classes +=================================================================================== + +Although this is a silly rule. Of course, APIs of editor classes need to be public for the other +modules. However, the other methods which are used only by helper classes in the editor module --the +methods may be crashed if called by the other modules because editor classes store and guarantee the +colleagues (e.g., ``Selection``) when it starts to handle an edit action (edit command or +operation)-- does not want to do it for the performance reason. Therefore, such methods are now +declared as protected methods and the caller classes are registered as friends. + +For solving this issue, we could split the editor classes one is exported and the other is not +exposed, and make the former to proxies and own the latter. However, this approach might cause +performance regressions and requires a lot of line changes (at least each method definition and +warning messages at the caller sides). Tracked in +`bug 1555916 <https://bugzilla.mozilla.org/show_bug.cgi?id=1555916>`__. + +Steps to handle one editor command or operation +=============================================== + +One edit command or operation is called "edit action" in the editor module. Handling it starts +when an XPCOM method or a public method which is named as ``*AsAction``. Those methods create +``AutoEditActionDataSetter`` in the stack first, then, call one of ``CanHandle()``, +``CanHandleAndMaybeDispatchBeforeInputEvent()`` or ``CanHandleAndFlushPendingNotifications()``. +If ``CanHandleAndMaybeDispatchBeforeInputEvent()`` causes dispatching ``beforeinput`` event and if +the event is consumed by the web app, it returns ``NS_ERROR_EDITOR_ACTION_CANCELED``. In this case, +the method can do anything due to the ``beforeinput`` event definition. + +At this time, ``AutoEditActionDataSetter`` stores ``Selection`` etc which are required for handling +the edit action in it and set ``EditorBase::mEditActionData`` to its address. Then all methods of +editor can access the objects via the pointer (typically wrapped in inline methods) and the lifetime +of the objects are guaranteed. + +Then, the methods call one or more edit-sub action handlers. E.g., when user types a character +with a non-collapsed selection range, editor needs to delete the selected content first and insert +the character there. For implementing this behavior, "insert text" edit action handler needs to call +"delete selection" sub-action handler and "insert text" sub-action handler. The sub-edit action +handlers are named as ``*AsSubAction``. + +The callers of edit sub-action handlers or the handlers themselves create ``AutoPlaceholderBatch`` +in the stack. This creates a placeholder transaction to make all transactions undoable with one +"undo" command. + +Then, each edit sub-action handler creates ``AutoEditSubActionNotifier`` in the stack and if it's +the topmost edit sub-action handling, ``OnStartToHandleTopLevelEditSubAction()`` is called at the +creation and ``OnEndHandlingTopLevelEditSubAction()`` is called at the destruction. The latter will +clean up the modified range, e.g., remove unnecessary empty nodes. + +Finally, the edit sub-actions does something while ``AutoEditSubActionNotifier`` is alive. Helper +methods of edit sub-action handlers are typically named as ``*WithTransaction`` because they are +done with transaction classes for making everything undoable. + +Don't update Selection immediately +================================== + +Changing the ranges of ``Selection`` is expensive (due ot validating new range, notifying new +selected or unselected frames, notifying selection listeners, etc), and retrieving current +``Selection`` ranges at staring to handle something makes the code statefull which is harder to +debug when you investigate a bug. Therefore, each method should return new caret position or +update ranges given as in/out parameter of ``AutoRangeArray``. ``Result<CaretPoint, nsresult>`` +is a good result type for the former, and the latter is useful style if the method needs to keep +``Selection`` similar to given ranges, e.g., when paragraphs around selection are changed to +different type of blocks. Finally, edit sub-action handler methods should update ``Selection`` +before destroying ``AutoEditSubActionNotifier`` whose post-processing requires ``Selection``. + +Don't add new things into OnEndHandlingTopLevelEditSubAction() +============================================================== + +When the topmost edit sub-action is handled, ``OnEndHandlingTopLevelEditSubAction`` is called and +it cleans up something in (or around) the modified range. However, this "post-processing" approach +makes it harder to change the behavior for fixing web-compat issues. For example, it deletes empty +nodes in the range, but if only some empty nodes are inserted intentionally, it doesn't have the +details and may unexpectedly delete necessary empty nodes. + +Instead, new things should be done immediately at or after modifying the DOM tree, and if it +requires to disable the post-processing, add new ``bool`` flag to +``EditorBase::TopLevelEditSubActionData`` and when it's set, make +``OnEndHandlingTopLevelEditSubAction`` stop doing something. + +Don't use NS_WARN_IF for checking NS_FAILED, isErr() and Failed() +================================================================= + +The warning messages like ``NS_FAILED(rv)`` does not help except the line number, and in the cases +of that we get web-compat reports, somewhere in the editor modules may get unexpected result. For +saving the investigation time of web-compat issues, each failure should warn which method call +failed, for example: + +.. code:: cpp + + nsresult rv = DoSomething(); + if (NS_FAILED(rv)) { + NS_WARNING("HTMLEditor::DoSomething() failed"); + return rv; + } + +These warnings will let you know the stack of failure in debug build. In other words, when you +investigate a web-compat issue in editor, you should do the steps to reproduce in debug build first. +Then, you'd see failure point stack in the terminal. + +Return NS_ERROR_EDITOR_DESTROYED when editor gets destroyed +=========================================================== + +The most critical error while an editor class method is running is what the editor instance is +destroyed by the web app. This can be checked with a call of ``EditorBase::Destroyed()`` and +if it returns ``true``, methods should return ``NS_ERROR_EDITOR_DESTROYED`` with stopping handling +anything. Then, all callers which handle the error result properly will stop handling too. +Finally, public methods should return ``EditorBase::ToGenericNSResult(rv)`` instead of exposing +an internal error of the editor module. + +Note that destroying the editor is intentional thing for the web app. Thus we should not throw +exception for this failure reason. Therefore, the public methods shouldn't return error. + +When you make a method return ``NS_ERROR_EDITOR_DESTROYED`` properly, you should mark the method +as ``[[nodiscard]]``. In other words, if you see ``[[nodiscard]]`` in method definition and it +returns ``nsresult`` or ``Result<*, nsresult>``, the method callers do not need to check +``Destroyed()`` by themselves. + +Use reference instead of pointer as far as possible +=================================================== + +When you create or redesign a method, it should take references instead of pointers if they take. +This rule forces that the caller to do null-check and this avoids a maybe unexpected case like: + +.. code:: cpp + + inline bool IsBRElement(const nsINode* aNode) { + return aNode && aNode->IsHTMLElement(nsGkAtoms::br); + } + + void DoSomethingExceptIfBRElement(const nsINode* aNode) { + if (IsBRElement(aNode)) { + return; + } + // Do something for non-BR element node. + } + +In this case, ``DoSomethingExceptIfBRElement`` expects that ``aNode`` is never ``nullptr`` but it +could be at least in build time. Using reference fixes this mistake at build time. + +Use ``EditorUtils`` or ``HTMLEditUtils`` for stateless methods +============================================================== + +When you create a new static method to the editor classes or a new inline method in cpp file which +defines the editor classes, please check if it's a common method which may be used from other +places in the editor module. If it's possible to be used only in ``HTMLEditor`` or its helper +classes, the method should be in ``HTMLEditUtils``. If it's possible be used in ``EditorBase`` or +``TextEditor`` or their helper classes, it should be in ``EditorUtils``. + +Don't use bool argument +======================= + +If you create a new method which take one or more ``bool`` arguments, use ``enum class`` instead +since ``true`` or ``false`` in the caller side is not easy to read. For example, you must not +be able to understand what this example mean: + +.. code:: cpp + + if (IsEmpty(aNode, true)) { + +For avoiding this issue, you should create new ``enum class`` for each. E.g., + +.. code:: cpp + + if (IsEmpty(aNode, TreatSingleBR::AsVisible)) { + +Basically, both ``enum class`` name and its value names explains what it means fluently. However, if +it's impossible, use ``No`` and ``Yes`` for the value like: + +.. code:: cpp + + if (DoSomething(aNode, OnlyIfEmpty::Yes)) { + +Don't use out parameters +======================== + +In most cases, editor methods meet error of low level APIs, thus editor methods usually return error +code. On the other hand, a lot of code need to return computed things, e.g., new caret position, +whether it's handled, ignored or canceled, a target node looked for, etc. We used ``nsresult`` for +the return value type and out parameters for the other results, but it makes callers scattering a +lot of auto variables and reusing them makes the code harder to understand. + +Now we can use ``mozilla::Result<Foo, nsresult>`` instead. diff --git a/editor/docs/EditorModuleStructure.rst b/editor/docs/EditorModuleStructure.rst new file mode 100644 index 0000000000..e179145a0d --- /dev/null +++ b/editor/docs/EditorModuleStructure.rst @@ -0,0 +1,219 @@ +####################### +Editor module structure +####################### + +This document explains the structure of the editor module and overview of classes. + +Introduction +============ + +This module implements the builtin editors of editable elements or documents, and this does **not** +implement the interface with DOM API and visual feedback of the editing UI. In other words, this +module implements DOM tree editors. + +Directories +=========== + +composer +-------- + +Previously, this directory contained "Composer" UI related code. However, currently, this +directory contains ``nsEditingSession`` and ``ComposerCommandsUpdater``. + +libeditor +--------- + +This is the main directory which contains "core" implementation of editors. + +spellchecker +------------ + +Despite of the directory name, implementation of the spellchecker is **not** here. This directory +contains only a bridge between editor classes and the spellchecker and serialized text of editable +content for spellchecking. + +txmgr +----- + +This directory contains transaction items and transaction classes. They were designed for generic +use cases, e.g., managing undo/redo of bookmarks/history of browser, etc, but they are used only by +the editor. + +Main classes +============ + +EditorBase +---------- + +``EditorBase`` class is an abstract class of editors. This inherits ``nsIEditor`` XPCOM interface, +implement common features which work with instance of classes, and exposed by +``mozilla/EditorBase.h``. + +TextEditor +---------- + +``TextEditor`` class is the implementation of plaintext editor which works with ``<input>`` and +``<textarea>``. Its exposed root is the host HTML elements, however, the editable root is an +anonymous ``<div>`` created in a native anonymous subtree under the exposed root elements. This +creates a ``Text`` node as the first child of the anonymous ``<div>`` and modify its data. If the text +data ends with a line-break, i.e., the last line is empty, append a ``<br>`` element for making the +empty last line visible. + +This also implements password editor. It works almost same as normal text editor, but each character +may be masked by masked character such as "●" or "*" by the layout module for the privacy. +Therefore, this manages masked/unmasked range of password and maybe making typed character +automatically after a while for mobile devices. + +This is exposed with ``mozilla/TextEditor.h``. + +Selection in TextEditor +^^^^^^^^^^^^^^^^^^^^^^^ + +Independent ``Selection`` and ``nsFrameSelection`` per ``<input>`` or ``<textarea>``. + +Lifetime of TextEditor +^^^^^^^^^^^^^^^^^^^^^^ + +Created when an editable ``<textarea>`` is created or a text-editable ``<input>`` element gets focus. +Note that the initialization may run asynchronously if it's requested when it's not safe to run +script. Destroyed when the element becomes invisible. Note that ``TextEditor`` is recreated when +every reframe of the host element. This means that when the size of ``<input>`` or ``<textarea>`` +is changed for example, ``TextEditor`` is recreated and forget undo/redo transactions, but takes +over the value, selection ranges and composition of IME from the previous instance. + +HTMLEditor +---------- + +``HTMLEditor`` class is the implementation of rich text editor which works with ``contenteditable``, +``Document.designMode`` and XUL ``<editor>``. Its instance is created per document even if the +document has multiple elements having ``contenteditable`` attribute. Therefore, undo/redo +transactions are shared in all editable regions. + +This is exposed with ``mozilla/HTMLEditor.h``. + +Selection in HTMLEditor +^^^^^^^^^^^^^^^^^^^^^^^ + +The instance for the ``Document`` and ``Window``. When an editable element gets focus, ``HTMLEditor`` +sets the ancestor limit of ``Selection`` to the focused element or the ``<body>`` of the ``Document``. +Then, ``Selection`` cannot cross boundary of the limiter element. + +Lifetime of HTMLEditor +^^^^^^^^^^^^^^^^^^^^^^ + +Created when first editable region is created in the ``Document``. Destroyed when last editable +region becomes non-editable. + +Currently, even while ``HTMLEditor`` is handling an edit command/operation (called edit action in +editor classes), each DOM mutation can be tracked with legacy DOM mutation events synchronously. +Thus, after changing the DOM tree from ``HTMLEditor``, any state could occur, e.g., the editor +itself may have been destroyed, the DOM tree have been modified, the ``Selection`` have been +modified, etc. This issue is tracked in +`bug 1710784 <https://bugzilla.mozilla.org/show_bug.cgi?id=1710784>`__. + + +EditorUtils +----------- + +This class has only static utility methods which are used by ``EditorBase`` or ``TextEditor`` and +may be used by ``HTMLEditor`` too. I.e., the utility methods which are used **not** only by +``HTMLEditor`` should be implemented in this class. + +Typically, sateless methods should be implemented as ``static`` methods of utility classes because +editor classes have too many methods and fields. + +This class is not exposed. + +HTMLEditUtils +------------- + +This class has only static utility methods which are used only by ``HTMLEditor``. + +This class is not exposed. + +AutoRangeArray +-------------- + +This class is a stack only class and intended to copy of normal selection ranges. In the new code, +`Selection` shouldn't be referred directly, instead, methods should take reference to this instance +and modify it. Finally, root caller should apply the ranges to `Selection`. Then, `HTMLEditor` +does not need to take care of unexpected `Selection` updates by legacy DOM mutation event listeners. + +This class is not exposed. + +EditorDOMPoint, EditorRawDOMPoint, EditorDOMPointInText, EditorRawDOMPointInText +-------------------------------------------------------------------------------- + +It represents a point in a DOM tree with one of the following: + +* Container node and offset in it +* Container node and child node in it +* Container node and both offset and child node in it + +In most cases, instances are initialized with a container and only offset or child node. Then, +when ``Offset()`` or ``GetChild()`` is called, the last one is "fixed". After inserting new child +node before the offset and/or the child node, ``IsSetAndValid()`` will return ``false`` since the +child node is not the child at the offset. + +If you want to keep using after modifying the DOM tree, you can make the instance forget offset or +child node with ``AutoEditorDOMPointChildInvalidator`` and ``AutoEditorDOMRangeChildrenInvalidator``. +The reason why the forgetting methods are not simply exposed is, ``Offset()`` and ``GetChild()`` +are available even after the DOM tree is modified to get the cached offset and child node, +additionally, which method may modify the DOM tree may be not clear for developers. Therefore, +creating a block only for these helper classes makes the updating point clearer. + +These classes are exposed with ``mozilla/EditorDOMPoint.h``. + +EditorDOMRange, EditorRawDOMRange, EditorDOMRangeInTexts, EditorRawDOMRangeInTexts +---------------------------------------------------------------------------------- + +It represents 2 points in a DOM tree with 2 ``Editor*DOMPoint(InText)``. Different from ``nsRange``, +the instances do not track the DOM tree changes. Therefore, the initialization is much faster than +``nsRange`` and can be in the stack. + +These classes are exposed with ``mozilla/EditorDOMPoint.h``. + +AutoTrackDOMPoint, AutoTrackDOMRange +------------------------------------ + +These methods updates ``Editor*DOMPoint(InText)`` or ``Editor*DOMRange(InTexts)`` at destruction +with applying the changes caused by the editor instance. In other words, they don't track the DOM +tree changes by the web apps like changes from legacy DOM mutation event listeners. + +These classes are currently exposed with ``mozilla/SelectionState.h``, but we should stop exposing +them. + +WSRunScanner +------------ + +A helper class of ``HTMLEditor``. This class scans previous or (inclusive) next visible thing from +a DOM point or a DOM node. This is typically useful for considering whether a `<br>` is visible or +invisible due to near a block element boundary, finding nearest editable character from caret +position, etc. However, the running cost is **not** cheap, thus if you find another way to consider +it simpler, use it instead, and also this does not check the actual style of the nodes (visible vs. +invisible, block vs. inline), thus you'd get unexpected result in tricky cases. + +This class is not exposed. + +WhiteSpaceVisibilityKeeper +-------------------------- + +A helper class of ``HTMLEditor`` to handle collapsible white-spaces as what user expected. This +class currently handles white-space normalization (e.g., when user inputs multiple collapsible +white-spaces, this replaces some of them to NBSPs), but the behavior is different from the other +browsers. We should re-implement this with emulating the other browsers' behavior as far as possible, +but currently it's put off due to not affecting UX (tracked in +`bug 1658699 <https://bugzilla.mozilla.org/show_bug.cgi?id=1658699>`__. + +This class is not exposed. + +\*Transaction +------------- + +``*Transaction`` classes represents a small transaction of updating the DOM tree and implements +"do", "undo" and "redo" of the update. + +Note that each class instance is created too many (one edit action may cause multiple transactions). +Therefore, each instance must be smaller as far as possible, and if you have an idea to collapse +multiple instances to one instance, you should fix it. Then, users can run Firefox with smaller +memory devices especially if the transaction is used in ``TextEditor``. diff --git a/editor/docs/IMEHandlingGuide.rst b/editor/docs/IMEHandlingGuide.rst new file mode 100644 index 0000000000..10f2d3b58e --- /dev/null +++ b/editor/docs/IMEHandlingGuide.rst @@ -0,0 +1,1091 @@ +================== +IME handling guide +================== + +This document explains how Gecko handles IME. + +Introduction +============ + +IME is an abbreviation of Input Method Editor. This is a technical term from +Windows but these days, this is used on other platforms as well. + +IME is a helper application of a user's text input. It handles native key +events before or after focused application (depending on the platform) and +creates a composition string (a.k.a. preedit string), suggests a list of what +the user attempts to input, commits composition string as a selected item off +the list and commits composition string without any conversion. IME is used by +Chinese, Japanese, Korean and Taiwan users for inputting Chinese characters +because the number of them is beyond thousands and cannot be input from the +keyboard directly. However, especially on mobile devices nowadays, IME is also +used for inputting Latin languages like autocomplete. Additionally, IME may be +used for handwriting systems or speech input systems on some platforms. + +If IME is available on focused elements, we call that state "enabled". If IME +is not fully available(i.e., user cannot enable IME), we call this state +"disabled". + +If IME is enabled but users use direct input mode (e.g., for inputting Latin +characters), we call it "IME is closed". Otherwise, we call it "IME is open". +(FYI: "open" is also called "active" or "turned on". "closed" is also called +"inactive" or "turned off") + +So, this document is useful when you're try to fix a bug for text input in +Gecko. + + +Composition string and clauses +============================== + +Typical Japanese IME can input two or more words into a composition string. +When a user converts from Hiragana characters to Chinese characters the +composition string, Japanese IME separates the composition string into multiple +clauses. For example, if a user types "watasinonamaehanakanodesu", it's +converted to Hiragana characters, "わたしのなまえはなかのです", automatically (In +the following screenshots, the composition string has a wavy underline and the +only one clause is called "raw input clause"). + +.. image:: inputting_composition_string.png + :alt: Screenshot of raw composition string which is inputting Roman + character mode of MS-IME (Japanese) + +.. image:: raw_composition_string.png + :alt: Screenshot of raw composition string whose all characters are Hiragana + character (MS-IME, Japanese) + +When a user presses ``Convert`` key, Japanese IME separates the composition +string as "わたしの" (my), "なまえは" (name is) and "なかのです" (Nakano). Then, +converts each clause with Chinese characters: "私の", "名前は" and "中野です" (In +the following screenshot each clause is underlined and not connected +adjacently. These clauses are called "converted clause"). + +.. image:: converted_composition_string.png + :alt: Screenshot of converted composition string (MS-IME, Japanese) + +If one or more clauses were not converted as expected, the user can choose one +of the clauses with Arrow keys and look for the expected result form the list +in the drop down menu (In the following screenshot, the clause with the thicker +underline is called "selected clause"). + +.. image:: candidatewindow.png + :alt: Screenshot of candidate window of MS-IME (Japanese) which converts the + selected clause + +Basically, composition string and each clause style is rendered by Gecko. And +the drop down menu is created by IME. + +Each clause is represented with selection in the editor. From chrome script, +you can check it with ``nsISelectionController``. In native code, you can +access it with either ``nsISelectionController`` or ``mozilla::SelectionType`` +(the latter is recommended because of type safer). And editor sets these IME +selections from ``mozilla::TextRangeType`` which are sent by +``mozilla::WidgetCompositionEvent`` as ``mozilla::TextRangeArray``. The +following table explains the mapping between them. + +.. table:: Selection types of each clause of composition string or caret + + +------------------------------------------------------------+---------------------------------------+-------------------------+-------------------------+ + | |`nsISelectionController`_ |`mozilla::SelectionType`_|`mozilla::TextRangeType`_| + +============================================================+=======================================+=========================+=========================+ + |Caret |``SELECTION_NORMAL`` |``eNormal`` |``eCaret`` | + +------------------------------------------------------------+---------------------------------------+-------------------------+-------------------------+ + |Raw text typed by the user |``SELECTION_IME_RAW_INPUT`` |``eIMERawClause`` |``eRawClause`` | + +------------------------------------------------------------+---------------------------------------+-------------------------+-------------------------+ + |Selected clause of raw text typed by the user |``SELECTION_IME_SELECTEDRAWTEXT`` |``eIMESelectedRawClause``|``eSelectedRawClause`` | + +------------------------------------------------------------+---------------------------------------+-------------------------+-------------------------+ + |Converted clause by IME |``SELECTION_IME_CONVERTEDTEXT`` |``eIMEConvertedClause`` |``eConvertedClause`` | + +------------------------------------------------------------+---------------------------------------+-------------------------+-------------------------+ + |Selected clause by the user or IME and also converted by IME|``SELECTION_IME_SELECTEDCONVERTEDTEXT``|``eIMESelectedClause`` |``eSelectedClause`` | + +------------------------------------------------------------+---------------------------------------+-------------------------+-------------------------+ + +Note that typically, "Selected clause of raw text typed by the user" isn't used +because when composition string is already separated to multiple clauses, that +means that the composition string has already been converted by IME at least +once. + +.. _nsISelectionController: https://searchfox.org/mozilla-central/source/dom/base/nsISelectionController.idl +.. _mozilla::SelectionType: https://searchfox.org/mozilla-central/source/dom/base/nsISelectionController.idl +.. _mozilla::TextRangeType: https://searchfox.org/mozilla-central/source/widget/TextRange.h + +Modules handling IME composition +================================ + +widget +------ + +Each widget handles native IME events and dispatches ``WidgetCompositionEvent`` +with ``mozilla::widget::TextEventDispatcher`` to represent the behavior of IME +in the focused editor. + +This is the only module that depends on the users platform. See also +`Native IME handlers`_ section for the detail of each platform's +implementation. + +.. note:: + + Android widget still does not use ``TextEventDispatcher`` to dispatch + ``WidgetCompositionEvents``, see + `bug 1137567 <https://bugzilla.mozilla.org/show_bug.cgi?id=1137567>`__. + +mozilla::widget::TextEventDispatcher +------------------------------------ + +This class is used by native IME handler(s) on each platform. This capsules the +logic to dispatch ``WidgetCompositionEvent`` and ``WidgetKeyboardEvent`` for +making the behavior on each platform exactly same. For example, if +``WidgetKeyboardEvent`` should be dispatched when there is a composition is +managed by this class in XP level. First of use, native IME handlers get the +rights to use ``TextEventDispatcher`` with a call of +``BeginNativeInputTransaction()``. Then, ``StartComposition()``, +``SetPendingComposition()``, ``FlushPendingComposition()``, +``CommitComposition()``, etc. are available if +``BeginNativeInputTransaction()`` return true. These methods automatically +manage composition state and dispatch ``WidgetCompositionEvent`` properly. + +This is also used by ``mozilla::TextInputProcessor`` which can emulates (or +implements) IME with chrome script. So, native IME handlers using this class +means that the dispatching part is also tested by automated tests. + +mozilla::WidgetCompositionEvent +------------------------------- + +Internally, ``WidgetCompositionEvent`` represents native IME behavior. Its +message is one of following values: + +eCompositionStart +^^^^^^^^^^^^^^^^^ + +This is dispatched at starting a composition. This represents a DOM +``compositionstart`` event. The mData value is a selected string at dispatching +the DOM event and it's automatically set by ``TextComposition``. + +eCompositionUpdate +^^^^^^^^^^^^^^^^^^ + +This is dispatched by ``TextComposition`` when an ``eCompositionChange`` will +change the composition string. This represents a DOM ``compositionupdate`` +event. + +eCompositionEnd +^^^^^^^^^^^^^^^ + +This is dispatched by ``TextComposition`` when an ``eCompositionCommitAsIs`` or +``eCompositionCommit`` event is dispatched. This represents a DOM +``compositionend`` event. + +eCompositionChange +^^^^^^^^^^^^^^^^^^ + +This is used internally only. This is dispatched at modifying a composition +string, committing a composition, changing caret position and/or changing +ranges of clauses. This represents a DOM text event which is not in any +standards. ``mRanges`` should not be empty only with this message. + +eCompositionCommitAsIs +^^^^^^^^^^^^^^^^^^^^^^ + +This is used internally only. This is dispatched when a composition is +committed with the string. The ``mData`` value should be always be an empty +string. This causes a DOM text event without clause information and a DOM +``compositionend`` event. + +eCompositionCommit +^^^^^^^^^^^^^^^^^^ + +This is used internally only. This is dispatched when a composition is +committed with specific string. The ``mData`` value is the commit string. This +causes a DOM text event without clause information and a DOM ``compositionend`` +event. + +.. table:: Table of event messages + + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + | |meaning of mData |who sets ``mData``? |``mRanges`` |representing DOM event| + +==========================+===========================================+===============================+=======================+======================+ + |``eCompositionStart`` |selected string before starting composition|``TextComposition`` |``nullptr`` |``compositionstart`` | + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + |``eCompositionUpdate`` |new composition string |``TextComposition`` |``nullptr`` |``compositionupdate`` | + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + |``eCompositionEnd`` |commit string |``TextComposition`` |``nullptr`` |``compositionend`` | + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + |``eCompositionChange`` |new composition string |widget (or ``TextComposition``)|must not be ``nullptr``|``text`` | + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + |``eCompositionCommitAsIs``|N/A (must be empty) |nobody |``nullptr`` |None | + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + |``eCompositionCommit`` |commit string |widget (or ``TextComposition``)|``nullptr`` |None | + +--------------------------+-------------------------------------------+-------------------------------+-----------------------+----------------------+ + +PresShell +--------- + +``PresShell`` receives the widget events and decides an event target from +focused document and element. Then, it sends the events and the event target to +``IMEStateManager``. + +mozilla::IMEStateManager +------------------------ + +``IMEStateManager`` looks for a ``TextComposition`` instance whose native IME +context is same as the widget' which dispatches the widget event. If there is +no proper ``TextComposition`` instance, it creates the instance. And it sends +the event to the ``TextComposition`` instance. + +Note that all instances of ``TextComposition`` are managed by +``IMEStateManager``. When an instance is created, it's registered to the list. +When composition completely ends, it's unregistered from the list (and released +automatically). + +mozilla::TextComposition +------------------------ + +``TextComposition`` manages a composition and dispatches DOM +``compositionupdate`` events. + +When this receives an ``eCompositionChange``, ``eCompositionCommit`` or +``eCompositionCommitAsIs`` event, it dispatches the event to the stored node +which was the event target of ``eCompositionStart`` event. Therefore, this +class guarantees that all composition events for a composition are fired on +same element. + +When this receives ``eCompositionChange`` or ``eCompositionCommit``, this +checks if new composition string (or committing string) is different from the +last data stored by the ``TextComposition``. If the composition event is +changing the composition string, the ``TextComposition`` instance dispatches +``WidgetCompositionEvent`` with ``eCompositionUpdate`` into the DOM tree +directly and modifies the last data. The ``eCompositionUpdate`` event will +cause a DOM ``compositionupdate`` event. + +When this receives ``eCompositionCommitAsIs`` or ``eCompositionCommit``, this +dispatches an ``eCompositionEnd`` event which will cause a DOM +``compositionend`` event after dispatching ``eCompositionUpdate`` event and/or +``eCompositionChange`` event if necessary. + +One of the other important jobs of this is, when a focused editor handles a +dispatched ``eCompositionChange`` event, this modifies the stored composition +string and its clause information. The editor refers the stored information for +creating or modifying a text node representing a composition string. + +And before dispatching ``eComposition*`` events, this class removes ASCII +control characters from dispatching composition event's data in the default +settings. Although, this can be disabled with +``"dom.compositionevent.allow_control_characters"`` pref. + +Finally, this class guarantees that requesting to commit or cancel current +composition to IME is perefored synchronously. See +`Forcibly committing composition`_ section for the detail. + +editor/libeditor +---------------- + +`mozilla::EditorEventListener <https://searchfox.org/mozilla-central/source/editor/libeditor/EditorEventListener.cpp>`__ +listens for trusted DOM ``compositionstart``, ``text`` and ``compositionend`` +events and notifies +`mozilla::EditorBase <https://searchfox.org/mozilla-central/source/editor/libeditor/EditorBase.cpp>`__ +and +`mozilla::TextEditor <https://searchfox.org/mozilla-central/source/editor/libeditor/TextEditor.cpp>`__ +of the events. + +When ``EditorBase`` receives an ``eCompositionStart`` +(DOM ``"compositionstart"``) event, it looks for a proper ``TextComposition`` +instance and stores it. + +When ``TextEditor`` receives an ``eCompositionChange`` (DOM ``"text"``) event, +it creates or modifies a text node which includes the composition string and +`mozilla::CompositionTransaction <https://searchfox.org/mozilla-central/source/editor/libeditor/CompositionTransaction.cpp>`__ +(it was called ``IMETextTxn``) sets IME selections for representing the clauses +of the composition string. + +When ``EditorBase`` receives an ``eCompositionEnd`` (DOM ``"compositionend"``) +event, it releases the stored ``TextComposition`` instance. + +nsTextFrame +----------- +``nsTextFrame`` paints IME selections. + +mozilla::IMEContentObserver +--------------------------- + +``IMEContentObserver`` observes various changes of a focused editor. When a +corresponding element of a ``TextEditor`` or ``HTMLEditor`` instance gets +focus, an instance is created by ``IMEStateManager``, then, starts to observe +and notifies ``widget`` of IME getting focus. When the editor loses focus, it +notifies ``widget`` of IME losing focus and stops observing everything. +Finally, it's destroyed by ``IMEStateManager``. + +This class observes selection changes (caret position changes), text changes of +a focused editor and layout changes (by reflow or scroll) of everything in the +document. It depends on the result of ``nsIWidget::GetIMEUpdatePreference()`` +what is observed. + +When this notifies ``widget`` of something, it needs to be safe to run +script because notifying something may cause dispatching one or more DOM events +and/or new reflow. Therefore, ``IMEContentObserver`` only stores which +notification should be sent to ``widget``. Then, +``mozilla::IMEContentObserver::IMENotificationSender`` tries to send the +pending notifications when it might become safe to do that. Currently, it's +tried: + +* after a native event is dispatched from ``PresShell::HandleEventInternal()`` +* when new focused editor receives DOM ``focus`` event +* when next refresh driver tick + +.. note:: + + The 3rd timing may not be safe actually, but it causes a lot of oranges of + automated tests. + +See also `Notifications to IME`_ section for the detail of sending +notifications. + +Currently, ``WidgetQueryContentEvent`` is handled via ``IMEContentObserver`` +because if it has a cache of selection, it can set reply of +``eQuerySelectedText`` event only with the cache. That is much faster than +using ``ContentEventHandler``. + +e10s support +============ + +Even when a remote process has focus, native IME handler in chrome process does +its job. So, there is process boundary between native IME handler and focused +editor. Unfortunately, it's not allowed to use synchronous communication from +chrome process to a remote process. This means that chrome process (and also +native IME and our native IME handler) cannot query the focused editor contents +directly. For fixing this issue, we have ``ContentCache`` classes around +process boundary. + +mozilla::ContentCache +--------------------- +This is a base class of ``ContentCacheInChild`` and ``ContentCacheInParent`` +and IPC-aware. This has common members of them including all cache data: + +``mText`` + Whole text in focused editor. This may be too big but IME may request all + text in the editor. + + If we can separate editor contents per paragraph, moving selection between + paragraphs generates pseudo focus move, we can reduce this size and runtime + cost of ``ContentEventHandler``. However, we've not had a plan to do that + yet. Note that Microsoft Word uses this hack. + +``mCompositionStart`` + Offset of composition string in ``mText``. When there is no composition, + this is ``UINT32_MAX``. + +``mSelection::mAnchor``, ``mSelection::mFocus`` + Offset of selection anchor and focus in ``mText``. + +``mSelection::mWritingMode`` + Writing mode at selection start. + +``mSelection::mAnchorCharRect``, ``mSelection::mFocusCharRect`` + Next character rectangle of ``mSelection::mAnchor`` and + ``mSelection::mFocus``. If corresponding offset is end of the editor + contents, its rectangle should be a caret rectangle. + + These rectangles shouldn't be empty rect. + +``mSelection::mRect`` + Unified character rectangle in selection range. When the selection is + collapsed, this should be caret rect. + +``mFirstRect`` + First character rect of ``mText``. When ``mText`` is empty string, this + should be caret rect. + +``mCaret::mOffset`` + Always same as selection start offset even when selection isn't collapsed. + +``mCaret::mRect`` + Caret rect at ``mCaret::mOffset``. If caret isn't actually exists, it's + computed with a character rect at the offset. + +``mTextRectArray::mStart`` + If there is composition, ``mStart`` is same as ``mCompositionStart``. + Otherwise, ``UINT32_MAX``. + +``mTextRectArray::mRects`` + Each character rectangle of composition string. + +``mEditorRect`` + The rect of editor element. + +mozilla::ContentCacheInChild +---------------------------- + +This exists only in remote processes. This is created as a member of +`PuppetWidget <https://searchfox.org/mozilla-central/source/widget/PuppetWidget.cpp>`__. +When ``PuppetWidget`` receives notifications to IME from ``IMEContentObserver`` +in the remote process, it makes this class modify its cached content. Then, +this class do that with ``WidgetQueryContentEvents``. Finally, ``PuppetWidget`` +sends the notification and ``ContentCacheInParent`` instance as +``ContentCache`` to its parent process. + +mozilla::ContentCacheInParent +----------------------------- + +This exists as a member of ``TabParent``. When ``TabParent`` receives +notification from corresponding remote process, it assigns +``ContentCacheInParent`` new ``ContentCache`` and post the notification to +``ContentCacheInParent``. If all sent ``WidgetCompositionEvents`` and +``WidgetSelectionEvents`` are already handled in the remote process, +``ContentCacheInParent`` sending the notifications to widget. + +And also this handles ``WidgetQueryContentEvents`` with its cache. Supported +event messages of them are: + +* ``eQuerySelectedText`` (only with ``SelectionType::eNormal``) +* ``eQueryTextContent`` +* ``eQueryTextRect`` +* ``eQueryCaretRect`` +* ``eQueryEditorRect`` + +Additionally, this does not support query content events with XP line breakers +but this must not be any problem since native IME handlers query contents with +native line breakers. + +``ContentCacheInParent`` also manages sent ``WidgetCompositionEvents`` and +``WidgetSelectionEvents``. After these events are handled in the remote +process, ``TabParent`` receives it with a call of +``RecvOnEventNeedingAckHandled()``. Then, it calls +``ContentCacheInParent::OnEventNeedingAckHandled()``. Finally, +``ContentCacheInParent`` flushes pending notifications. + +How do mozilla::TextComposition and mozilla::IMEStateManager work in e10s mode? +------------------------------------------------------------------------------- +In remote process, they work as non-e10s mode. On the other hand, they work +specially in parent process. + +When ``IMEStateManager`` in parent process receives ``eCompositionStart``, it +creates ``TextComposition`` instance normally. However, if the event target has +remote contents, ``TextComposition::DispatchCompositionEvent()`` directly sends +the event to the remote process instead of dispatching the event into the +target DOM tree in the process. + +That means that even in a parent process, anybody can retrieve +``TextComposition`` instance, but it just does nothing in parent process. + +``IMEStateManager`` works more complicated because ``IMEStateManager`` in each +process need to negotiate about owner ship of managing input context. + +When a remote process gets focus, temporarily, ``IMEStateManager`` in parent +process disables IME in the widget. After that, ``IMEStateManager`` in the +remote process will set proper input context for the focused editor. At this +time, ``IMEStateManager`` in the parent process does nothing. Therefore, +``IMEContentObserver`` is never created while a remote process has focus. + +When a remote process loses focus, ``IMEStateManager`` in parent process +notifies ``IMEStateManager`` in the remote process of +"Stop IME state management". When ``IMEStateManager::StopIMEStateManagement()`` +is called in the remote process by this, the ``IMEStateManager`` forgets all +focus information (i.e., that indicates nobody has focus). + +When ``IMEStateManager`` in parent process is notified of pseudo focus move +from or to menubar while a remote process has focus, it notifies the remote +process of "Menu keyboard listener installed". Then, ``TabChild`` calls +``IMEStateManager::OnInstalledMenuKeyboardListener()`` in the remote process. + +Style of each clause +-------------------- + +The style of each IME selection is managed by +`LookAndFeel <https://searchfox.org/mozilla-central/source/widget/LookAndFeel.h>`__ +class per platform. Therefore, it can be overridden by prefs. + +Background color, foreground color (text color) and underline color can be +specified with following prefs. The values must be string of "#rrggbb" format. + +* ``ui.IMERawInputBackground`` +* ``ui.IMERawInputForeground`` +* ``ui.IMERawInputUnderline`` +* ``ui.IMESelectedRawTextBackground`` +* ``ui.IMESelectedRawTextForeground`` +* ``ui.IMESelectedRawTextUnderline`` +* ``ui.IMEConvertedTextBackground`` +* ``ui.IMEConvertedTextForeground`` +* ``ui.IMEConvertedTextUnderline`` +* ``ui.IMESelectedConvertedTextBackground`` +* ``ui.IMESelectedConvertedTextForeground`` +* ``ui.IMESelectedConvertedTextUnderline`` + +Underline style can be specified with the following prefs. The values are +integer, 0: none, 1: dotted, 2: dashed, 3: solid, 4: double, 5: wavy (The +values same as ``mozilla::StyleTextDecorationStyle`` defined in +`nsStyleConsts.h <https://searchfox.org/mozilla-central/source/layout/style/nsStyleConsts.h>`__). + +* ``ui.IMERawInputUnderlineStyle`` +* ``ui.IMESelectedRawTextUnderlineStyle`` +* ``ui.IMEConvertedTextUnderlineStyle`` +* ``ui.IMESelectedConvertedTextUnderlineStyle`` + +Underline width can be specified with ``"ui.IMEUnderlineRelativeSize"`` pref. +This affects all types of clauses. The value should be 100 or 200. 100 means +normal width, 200 means double width. + +On some platforms, IME may support its own style for each clause. Currently, +this feature is supported in TSF mode of Windows and on Linux. The style +information is stored in ``TextRangeStyle`` which is defined in +`TextRange.h <https://searchfox.org/mozilla-central/source/widget/TextRange.h>`__. +It's a member of ``TextRange``. ``TextRange`` is stored in ``mRanges`` of +``WidgetCompositionEvent`` only when its message is ``eCompositionChange``. + +Lifetime of composition string +============================== + +When native IME notifies Gecko of starting a composition, a widget dispatches +``WidgetCompositionEvent`` with ``eCompositionStart`` which will cause a DOM +``compositionstart`` event. + +When native IME notifies Gecko of a composition string change, a caret position +change and/or a change of length of clauses, a widget dispatches +``WidgetCompositionEvent`` with ``eCompositionChange`` event. It will cause a +DOM ``compositionupdate`` event when composition string is changing. That is +dispatched by ``TextComposition`` automatically. After that when the widget and +``PresShell`` of the focused editor have not been destroyed yet, the +``eCompositionChange`` will cause a DOM text event which is not in any web +standards. + +When native IME notifies Gecko of the ending of a composition, a widget +dispatches ``WidgetCompositionEvent`` with ``eCompositionCommitAsIs`` or +``eCompositionCommit``. If the committing string is different from the last set +of data (i.e., if the event message is ``eCompositionCommit``), +``TextComposition`` dispatches a DOM ``compositionupdate`` event. After that, +when the widget and ``PresShell`` of the focused editor have not been destroyed +yet, an ``eCompositionChange`` event dispatched by ``TextComposition``, that +causes a DOM text event. Finally, if the widget and PresShell of the focused +editor has not been destroyed yet too, ``TextComposition`` dispatches an +``eCompositionEnd`` event which will cause a DOM compositionend event. + +Limitation of handling composition +================================== + +Currently, ``EditorBase`` touches undo stack at receiving every +``WidgetCompositionEvent``. Therefore, ``EditorBase`` requests to commit +composition when the following cases occur: + +* The editor loses focus +* The caret is moved by mouse or Javascript +* Value of the editor is changed by Javascript +* Node of the editor is removed from DOM tree +* Somethings object is modified in an HTML editor, e.g., resizing an image +* Composition string is moved to a different position which is specified by + native IME (e.g., only a part of composition is committed) + +In the future, we should fix this limitation. If we make ``EditorBase`` not +touch undo stack until composition is committed, some of the cases must be +fixed. + +Notifications to IME +==================== + +XP part of Gecko uses ``nsIWidget::NotifyIME()`` for notifying ``widget`` of +something useful to handle IME. Note that some of them are notified only when +``nsIWidget::GetIMEUpdatePreference()`` returns flags which request the +notifications. + +``NOTIFY_IME_OF_TEXT_CHANGE``, ``NOTIFY_IME_OF_SELECTION_CHANGE``, +``NOTIFY_IME_OF_POSITION_CHANGE`` and +``NOTIFY_IME_OF_COMPOSITION_EVENT_HANDLED`` are always sent by following order: + +1. ``NOTIFY_IME_OF_TEXT_CHANGE`` +2. ``NOTIFY_IME_OF_SELECTION_CHANGE`` +3. ``NOTIFY_IME_OF_POSITION_CHANGE`` +4. ``NOTIFY_IME_OF_COMPOSITION_EVENT_HANDLED`` + +If sending one of above notifications causes higher priority notification, the +sender should abort to send remaining notifications and restart from high +priority notification again. + +Additionally, all notifications except ``NOTIFY_IME_OF_BLUR`` should be sent +only when it's safe to run script since the notification may cause querying +content and/or dispatching composition events. + +NOTIFY_IME_OF_FOCUS +------------------- + +When an editable editor gets focus and ``IMEContentObserver`` starts to observe +it, this is sent to widget. This must be called after the previous +``IMEContentObserver`` notified widget of ``NOTIFY_IME_OF_BLUR``. + +Note that even if there are pending notifications, they are canceled when +``NOTIFY_IME_OF_FOCUS`` is sent since querying content with following +notifications immediately after getting focus does not make sense. The result +is always same as the result of querying contents at receiving this +notification. + +NOTIFY_IME_OF_BLUR +------------------ + +When an ``IMEContentObserver`` instance ends observing the focused editor, this +is sent to ``widget`` synchronously because assumed that this notification +causes neither query content events nor composition events. + +If ``widget`` wants notifications even while all windows are inactive, +``IMEContentObserver`` doesn't end observing the focused editor. I.e., in this +case, ``NOTIFY_IME_OF_FOCUS`` and ``NOTIFY_IME_OF_BLUR`` are not sent to +``widget`` when a window which has a composition is being activated or +inactivated. + +When ``widget`` wants notifications during inactive, ``widget`` includes +``NOTIFY_DURING_DEACTIVE`` to the result of +``nsIWidget::GetIMEUpdatePreference()``. + +If this notification is tried to sent before sending ``NOTIFY_IME_OF_FOCUS``, +all pending notifications and ``NOTIFY_IME_OF_BLUR`` itself are canceled. + +NOTIFY_IME_OF_TEXT_CHANGE +------------------------- + +When text of focused editor is changed, this is sent to ``widget`` with a range +of the change. But this is sent only when result of +``nsIWidget::GetIMEUpdatePreference()`` includes ``NOTIFY_TEXT_CHANGE``. + +If two or more text changes occurred after previous +``NOTIFY_IME_OF_TEXT_CHANGE`` or ``NOTIFY_IME_OF_FOCUS``, the ranges of all +changes are merged. E.g., if first change is from ``1`` to ``5`` and second +change is from ``5`` to ``10``, the notified range is from ``1`` to ``10``. + +If all merged text changes were caused by composition, +``IMENotification::mTextChangeData::mCausedOnlyByComposition`` is set to true. +This is useful if native IME handler wants to ignore all text changes which are +expected by native IME. + +If at least one text change of the merged text changes was caused by current +composition, +``IMENotification::mTextChangeData::mIncludingChangesDuringComposition`` is set +to true. This is useful if native IME handler wants to ignore delayed text +change notifications. + +If at least one text change of the merged text changes was caused when there +was no composition, +``IMENotification::mTextChangeData::mIncludingChangesWithoutComposition`` is +set to true. + +NOTIFY_IME_OF_SELECTION_CHANGE +------------------------------ + +When selection (or caret position) is changed in focused editor, widget is +notified of this. + +If the last selection change was occurred by a composition event event +handling, ``IMENotification::mSelectionChangeData::mCausedByComposition`` is +set to true. This is useful if native IME handler wants to ignore the last +selection change which is expected by native IME. + +If the last selection change was occurred by an ``eSetSelection`` event, +``IMENotification::mSelectionChangeData::mCausedBySelectionEvent`` is set to +true. This is useful if native IME handler wants to ignore the last selection +change which was requested by native IME. + +If the last selection is occurred during a composition, +``IMENotification::mSelectionChangeData::mOccurredDuringComposition`` is set to +true. This is useful if native IME handler wants to ignore the last selection +change which occurred by web application's ``compositionstart`` or +``compositionupdate`` event handler before inserting composition string. + +NOTIFY_IME_OF_POSITION_CHANGE +----------------------------- + +When reflow or scroll occurs in the document, this is sent to widget, but this +is sent only when result of ``nsIWidget::GetIMEUpdatePreference()`` includes +``NOTIFY_POSITION_CHANGE``. + +This might be useful to update a candidate window position or something. + +NOTIFY_IME_OF_COMPOSITION_EVENT_HANDLED +--------------------------------------- + +After ``TextComposition`` handles ``eCompositionStart``, +``eCompositionChange``, ``eComposiitionCommit`` or ``eCompositionCommitAsIs``, +this notification is sent to widget. This might be useful to update a candidate +window position or something. + +NOTIFY_IME_OF_MOUSE_BUTTON_EVENT +-------------------------------- + +When a ``mousedown`` event or a ``mouseup`` event is fired on a character in a +focused editor, this is sent to widget. But this is sent only when result of +``nsIWidget::GetIMEUpdatePreference()`` includes +``NOTIFY_MOUSE_BUTTON_EVENT_ON_CHAR``. This is sent with various information. +See ``IMENotification::mMouseButtonEventData`` in +`IMEData.h <https://searchfox.org/mozilla-central/source/widget/IMEData.h>`__ +for the detail. + +If native IME supports mouse button event handling, ``widget`` should notify +IME of mouse button events with this. If IME consumes an event, ``widget`` +should return ``NS_SUCCESS_EVENT_CONSUMED`` from ``nsIWidget::NotifyIME()``. +Then, ``EditorBase`` doesn't handle the mouse event. + +Note that if a ``mousedown`` event or a ``mouseup`` event is consumed by a web +application (before a focused editor handles it), this notification is not sent +to ``widget``. This means that web applications can handle mouse button events +before IME. + +Requests to IME +=============== + +XP part of Gecko can request IME to commit or cancel composition. This must be +requested via ``IMEStateManager::NotifyIME()``. Then, ``IMEStateManager`` looks +for a proper ``TextComposition`` instance. If it's found, +``TextComposition::RequestToCommit()`` for calling ``nsIWidget::NotifyIME()`` +and handles some extra jobs. + +widget should call the proper native API if it's available. Even if commit or +canceling composition does not occur synchronously, widget doesn't need to +emulate it since ``TextComposition`` will emulate it automatically. In other +words, widget should only request to commit or cancel composition to IME. + +REQUEST_TO_COMMIT_COMPOSITION +----------------------------- + +A request to commit current composition to IME. See also following +"`Forcibly committing composition`_" section for additional information. + +REQUEST_TO_CANCEL_COMPOSITION +----------------------------- + +A request to cancel current composition to IME. In other words, a request to +commit current composition with an empty string. + +Forcibly committing composition +=============================== + +When ``TextComposition::RequestToCommit()`` calls ``nsIWidget::NotifyIME()``, +it guarantees synchronous commit or canceling composition. + +In order to put it into practice, we need to handle the following four +scenarios: + +The composition is committed with non-empty string synchronously +---------------------------------------------------------------- + +This is the most usual case. In this case, ``TextComposition`` handles +``WidgetCompositionEvent`` instances during a request normally. However, in a +remote process in e10s mode, this case never occurs since requests to native +IME is handled asynchronously. + +The composition is not committed synchronously but later +-------------------------------------------------------- + +This is the only case in a remote process in e10s mode or occurs on Linux even +in non-e10s mode if the native IME is iBus. The callers of +``NotifyIME(REQUEST_TO_COMMIT_COMPOSITION)`` may expect that composition string +is committed immediately for their next job. For such a case, +``TextComposition::RequestToCommit()`` synthesizes DOM composition events and a +DOM text event for emulating to commit composition synchronously. Additionally, +``TextComposition`` ignores committing events which are dispatched by widget +when the widget receives native IME events. + +In this case, using the last composition string as commit string. + +However, if the last composition string is only an ideographic space (fullwidth +space), the composition string may be a placeholder of some old Chinese IME on +Windows. + +.. image:: ChangJie.png + :alt: aScreenshot of ChangJie (Traditional Chinese IME) which puts an + ideographic space into composition string for placeholder + +In this case, although, we should not commit the placeholder character because +it's not a character which the user wanted to input but we commit it as is. The +reason is, inputting an ideographic space causes a composition. Therefore, we +cannot distinguish if committing composition is unexpected. If the user uses +such old Chinese IME, ``"intl.ime.remove_placeholder_character_at_commit"`` +pref may be useful but we don't support them anymore in default settings +(except if somebody will find a good way to fix this issue). + +The composition is committed synchronously but with empty string +---------------------------------------------------------------- + +This case may occur on Linux or with some IME on other platforms. If a web +application implements autocomplete, committing with different strings +especially an empty string it might cause confusion. + +In this case, TextComposition overwrites the commit string of +``eCompositionChange`` event dispatched by widget. However, if the last +composition string is only an ideographic space, it shouldn't be committed. See +the previous case. + +Note that this case doesn't work as expected when composition is in a remote +process in e10s mode. + +The composition is not committed +-------------------------------- + +On Linux, there is no API to request commit or canceling composition forcibly. +Instead, Gecko uses ``gtk_im_context_reset()`` API for this purpose because +most IME cancel composition with it. But there are some IMEs which do nothing +when Gecko calls it. + +If this occurs, Gecko should restart composition with a DOM +``compositionstart`` event , a DOM ``compositionupdate`` event and a DOM +``text`` event at caret position. + +.. note:: + + This issue hasn't been supported yet. + +IME state management +==================== + +IME is a text input system. It means that except when a user wants to input +some text, IME shouldn't be available. For example, pressing the space key to +attempt scrolling a page may be consumed and prevented by IME. Additionally, +password editors need to request special behavior with IME. + +For solving this issue, Gecko sets the proper IME state at DOM focus change. + +First, when a DOM node gets focus, nsFocusManager notifies ``IMEStateManager`` +of the new focused node (calls ``IMEStateManager::OnChangeFocus()``). +``IMEStateManager`` asks desired IME state by calling +``nsIContent::GetDesiredIMEState()`` of the node. If the node owns +``TextEditor`` instance, it asks for the desired IME state from the editor and +returns the result. + +Next, ``IMEStateManager`` initializes ``InputContext`` (defined in +`IMEData.h <https://searchfox.org/mozilla-central/source/widget/IMEData.h>`__) +with the desired IME state and node information. Then, it calls +``nsIWidget::SetInputContext()`` with the ``InputContext``. + +Finally, widget stores the InputContext and enables or disables IME if the +platform has such an API. + +InputContext +------------ + +InputContext is a struct. Its ``mIMEState``, ``mHTMLInputType``, +``mHTMLInputInputMode`` and ``mActionHint`` are set at +``nsIWidget::SetInputContext()`` called. + +mIMEState +^^^^^^^^^ +IME state has two abilities. One is enabled state: + +ENABLED +""""""" + +This means IME is fully available. E.g., when an editable element such as +``<input type="text">``, ``<textarea>`` or ``<foo contenteditable>`` has focus. + +DISABLED +"""""""" + +This means IME is not available. E.g., when a non-editable element has focus or +no element has focus, the desired IME state is ``DISABLED``. + +PASSWORD +"""""""" + +This means IME state should be the same as the state when a native password +field has focus. This state is set only when +``<input type="password"> (ime-mode: auto;)``, +``<input type="text" style="ime-mode: disabled;">`` or +``<textarea style="ime-mode: disabled;">``. + +The other is IME open state: + +DONT_CHANGE_OPEN_STATE +"""""""""""""""""""""" + +The open state of IME shouldn't be changed. I.e., Gecko should keep the last +IME open state. + +OPEN +"""" +Open IME. This is specified only when ime-mode of the new focused element is +``active``. + +CLOSE +""""" +Close IME. This is specified only when ime-mode of the new focused element is +``inactive``. + +.. note:: + + E.g., on Linux, applications cannot manage IME open state. On such + platforms, this is ignored. + +.. note:: + + IME open state should be changed only when ``nsIWidget::SetInputContext()`` + is called at DOM focus change because changing IME open state while an + editor has focus makes users confused. The reason why + ``nsIWidget::SetInputContext()`` is called is stored in + ``InputContextAction::mCause``. + +How does Gecko disable IME in IMM mode on Windows +""""""""""""""""""""""""""""""""""""""""""""""""" + +Every window on Windows is associated an ``IMContext``. When Gecko disables +IME, +`mozilla::widget::IMEHandler <https://searchfox.org/mozilla-central/source/widget/windows/WinIMEHandler.cpp>`__::SetInputContext() +disassociates the context from the window. + +How does Gecko disable IME in TSF mode on Windows +""""""""""""""""""""""""""""""""""""""""""""""""" + +`mozilla::widget::TSFTextStore <https://searchfox.org/mozilla-central/source/widget/windows/TSFTextStore.cpp>`__ +sets focus to a dummy context which disables the keyboard. + +How does Gecko disable IME on Mac +""""""""""""""""""""""""""""""""" + +`mozilla::widget::TextInputHandler <https://searchfox.org/mozilla-central/source/widget/cocoa/TextInputHandler.mm>`__::HandleKeyDownEvent() +doesn't call focused view's interpretKeyEvents. This prevents native key events +to be passed to IME. + +How does Gecko disable IME on GTK +""""""""""""""""""""""""""""""""" + +`mozilla::widget::IMContextWrapper <https://searchfox.org/mozilla-central/source/widget/gtk/IMContextWrapper.cpp>`__ +sets focus to a dummy context which doesn't have IME composition. + +How does Gecko disable IME on Android +""""""""""""""""""""""""""""""""""""" + +? + +mHTMLInputType +^^^^^^^^^^^^^^ + +The value is a string representing the focused editor. + +``"text"``, ``"password"``, ``"number"``, etc. + When an ``<input>`` element gets focus, the value is the type of the input + element. + +``"textarea"`` + When a ``<textarea>`` element gets focus, the value is ``"textarea"``. + +``""`` + When an HTML editor (an element whose ``contenteditable`` attribute is + ``true`` or document whose ``designMode`` is ``"on"``) gets focus, the + value is empty. And also, when the other elements get focus. + +mHTMLInputMode +^^^^^^^^^^^^^^ + +The value is ``inputmode`` attribute value of the focused editor. + +mActionHint +^^^^^^^^^^^ + +The value is ``enterkeyhint`` attribute value of the focused editor when +``"dom.forms.enterkeyhint"`` pref is true. This is useful for deciding the +caption for the submit button in virtual keyboard. E.g., the value could be +``"Go"``, ``"Next"`` or ``"Search"``. + +Native IME handlers +=================== + +Following classes handles IME on each platform: + +Windows +------- + +`mozilla::widget::IMEHandler`__ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This class manages input method context of each window and makes ``IMMHandler`` +or ``TSFTextStore`` work with active IME and focused editor. This class has +only static members, i.e., never created its instance. + +__ https://searchfox.org/mozilla-central/source/widget/windows/WinIMEHandler.cpp + +`mozilla::widget::IMMHandler`__ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This class is used when TSF mode is disabled by pref (``"intl.tsf.enabled"`` +since 108, formerly named ``"intl.tsf.enable"``) or active IME is for IMM +(i.e., not TIP for TSF). + +This class handles ``WM_IME_*`` messages and uses ``Imm*()`` API. This is a +singleton class since Gecko supports only on IM context in a process. +Typically, a process creates windows with default IM context. Therefore, this +design is enough (ideally, an instance should be created per IM context, +though). The singleton instance is created when it becomes necessary. + +__ https://searchfox.org/mozilla-central/source/widget/windows/IMMHandler.cpp + +`mozilla::widget::TSFTextStore`__ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This class handles IME events in TSF mode and when TIP (IME implemented with +TSF) is active. This instances are created when an editable element gets focus +and released when it loses focus. + +``TSFTextStore`` implements some COM interfaces which is necessary to work with +TIP. And similarly, there is a singleton class, ``TSFStaticSink``, to observe +active TIP changes. + +TSF is the most complicated IME API on all platforms, therefore, design of this +class is also very complicated. + +FIrst, TSF/TIP requests to lock the editor content for querying or modifying +the content or selection. However, web standards don't have such mechanism. +Therefore, when it's requested, ``TSFTextStore`` caches current content and +selection with ``WidgetQueryContentEvent``. Then, it uses the cache to reply to +query requests, and modifies the cache as they requested. At this time, +``TSFTextStore`` saves the requests of modification into the queue called +``PendingAction``. Finally, after unlocking the contents, it flushes the +pending actions with dispatches ``WidgetCompositionEvent``s via +``TextEventDispatcher``. + +Then, ``IMEContentObserver`` will notify some changes caused by the dispatched +``WidgetCompositionEvents`` (they are notified synchronously in chrome or +non-e10s mode, but asynchronously from a remote process in e10s mode). At this +time, ``TSFTextStore`` may receive notifications which indicates web +application changes the content differently from cache in ``TSFTextStore``. +However, ``TSFTextStore`` ignores such fact temporarily until the composition +is finished completely. The reason is that, notifying unexpected text or +selection changes to TSF and/or TIP during composition may behave them odd. + +When a composition is committed and it receives +``NOTIFY_IME_OF_COMPOSITION_EVENT_HANDLED``, ``TSFTextStore`` clears the cache +of contents and notifying TSF of merged text changes and the last selection +change if they are not caused by composition. By this step, TSF and TIP may +sync its internal cache with actual contents. + +Note that if new composition is started before +``NOTIFY_IME_OF_COMPOSITION_EVENT_HANDLED`` notification, ``TSFTextStore`` +handles the a composition with cached contents which may be different from +actual contents. So, e.g., reconversion around caret may not work as unexpected +in such case, but we don't have a good solution for this issue. + +On the other hand, ``TSFTextStore`` cannot cache character rectangles since if +there are a lot of characters, caching the rectangles require a lot of CPU cost +(to compute each rect) and memory. Therefore, ``TSFTextStore`` will use +insertion point relative query for them +`bug 1286157 <https://bugzilla.mozilla.org/show_bug.cgi?id=1286157>`__. Then, +it can retrieve expected character's rect even if the cache of ``TSFTextStore`` +is different from the actual contents because TIP typically needs caret +position's character rect (for a popup to indicate current input mode or next +word suggestion list) or first character rect of the target clause of current +composition (for a candidate list window of conversion). + +__ https://searchfox.org/mozilla-central/source/widget/windows/TSFTextStore.cpp + +Mac +--- + +Both IME and key events are handled in +`TextInputHandler.mm <https://searchfox.org/mozilla-central/source/widget/cocoa/TextInputHandler.mm>`__. + +``mozilla::widget::TextInputHandlerBase`` is the most base class. +``mozilla::widget::IMEInputHandler`` inherits ``TextInputHandlerBase`` and +handles IME related events. ``mozilla::widget::TextInputHandler`` inherits +``TextInputHandlerBase`` and implements ``NSTextInput`` protocol of Cocoa. Its +instance is created per +`nsChildView <https://searchfox.org/mozilla-central/source/widget/cocoa/nsChildView.mm>`__ +instance. + +GTK +--- + +`mozilla::widget::IMContextWrapper <https://searchfox.org/mozilla-central/source/widget/gtk/IMContextWrapper.cpp>`__ +handles IME. The instance is created per top level window. + +Android +------- + +`org.mozilla.geckoview.GeckoEditable <https://searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java>`__ handles native IME events and `mozilla::widget::GeckoEditableSupport <https://searchfox.org/mozilla-central/source/widget/android/GeckoEditableSupport.cpp>`__ +dispatches ``Widget*Event``. diff --git a/editor/docs/candidatewindow.png b/editor/docs/candidatewindow.png Binary files differnew file mode 100644 index 0000000000..b11822a2d8 --- /dev/null +++ b/editor/docs/candidatewindow.png diff --git a/editor/docs/converted_composition_string.png b/editor/docs/converted_composition_string.png Binary files differnew file mode 100644 index 0000000000..fe61095bfb --- /dev/null +++ b/editor/docs/converted_composition_string.png diff --git a/editor/docs/index.rst b/editor/docs/index.rst new file mode 100644 index 0000000000..f272dd6684 --- /dev/null +++ b/editor/docs/index.rst @@ -0,0 +1,12 @@ +Editor +====== + +This collection of linked pages contains the document for the +editor. The documents live in editor/docs/. + +.. toctree:: + :maxdepth: 1 + + EditorModuleStructure + EditorModuleSpecificRules + IMEHandlingGuide diff --git a/editor/docs/inputting_composition_string.png b/editor/docs/inputting_composition_string.png Binary files differnew file mode 100644 index 0000000000..0df68b90c1 --- /dev/null +++ b/editor/docs/inputting_composition_string.png diff --git a/editor/docs/raw_composition_string.png b/editor/docs/raw_composition_string.png Binary files differnew file mode 100644 index 0000000000..afa3c0062a --- /dev/null +++ b/editor/docs/raw_composition_string.png |
