diff options
Diffstat (limited to 'browser/components/touchbar/docs/index.rst')
| -rw-r--r-- | browser/components/touchbar/docs/index.rst | 364 |
1 files changed, 364 insertions, 0 deletions
diff --git a/browser/components/touchbar/docs/index.rst b/browser/components/touchbar/docs/index.rst new file mode 100644 index 0000000000..6f11988900 --- /dev/null +++ b/browser/components/touchbar/docs/index.rst @@ -0,0 +1,364 @@ +Touch Bar +========= + +The Touch Bar is a hardware component on some MacBook Pros released from 2016. +It is a display above the keyboard that allows more flexible types of +input than is otherwise possible with a normal keyboard. Apple offers Touch Bar +APIs so developers can extend the Touch Bar to display inputs specific to their +application. Firefox consumes these APIs to offer a customizable row of inputs +in the Touch Bar. + +In Apple's documentation, the term "the Touch Bar" refers to the hardware. +The term "a Touch Bar" refers not to the hardware but to a collection of inputs +shown on the Touch Bar. This means that there can be multiple "Touch Bars" that +switch out as the user switches contexts. The same naming convention is used in +this document. + +In this document and in the code, the word "input" is used to refer to +an interactive element in the Touch Bar. It is often interchangeable with +"button", but "input" can also refer to any element displayed in the Touch Bar. + +The Touch Bar should never offer functionality unavailable to Firefox users +without the Touch Bar. Most macOS Firefox users do not have the Touch Bar and +some choose to disable it. Apple's own `Human Interface Guidelines`_ (HIG) +forbids this kind of Touch Bar functionality. Please read the HIG for more +design considerations before you plan on implementing a new Touch Bar feature. + +If you have questions about the Touch Bar that are not answered in this +document, feel free to reach out to `Harry Twyford`_ (:harry on Slack). +He wrote this document and Firefox's initial Touch Bar implementation. + +.. _Human Interface Guidelines: https://developer.apple.com/design/human-interface-guidelines/macos/touch-bar/touch-bar-overview/ + +.. _Harry Twyford: mailto:harry@mozilla.com + +.. contents:: Table of Contents + +Overview +~~~~~~~~ + +Firefox's Touch Bar implementation is equal parts JavaScript and Cocoa +(Objective-C++). The JavaScript code lives in ``browser/components/touchbar`` +and the Cocoa code lives in ``widget/cocoa``, mostly in ``nsTouchBar.mm``. The +Cocoa code is a consumer of Apple's Touch Bar APIs and defines what types of +Touch Bar inputs are available to its own consumers. The JS code in +``browser/components/touchbar`` provides services to ``nsTouchBar.mm`` and +defines what inputs the user actually sees in the Touch Bar. There is two-way +communication between the JS and the Cocoa: the Cocoa code asks the JS what +inputs it should display, and the JS asks the Cocoa code to update those inputs +when needed. + +JavaScript API +~~~~~~~~~~~~~~ + +``browser/components/touchbar/MacTouchBar.sys.mjs`` defines what specific inputs are +available to the user, what icon they will have, what action they will perform, +and so on. Inputs are defined in the ``gBuiltInInputs`` object `in that file`_. +When creating a new object in ``gBuiltInInputs``, the available properties are +documented in the JSDoc for ``TouchBarInput``: + +.. highlight:: JavaScript +.. code:: + + /** + * A representation of a Touch Bar input. + * @param {string} input.title + * The lookup key for the button's localized text title. + * @param {string} input.image + * A URL pointing to an SVG internal to Firefox. + * @param {string} input.type + * The type of Touch Bar input represented by the object. + * Must be a value from kInputTypes. + * @param {Function} input.callback + * A callback invoked when a touchbar item is touched. + * @param {string} [input.color] + * A string in hex format specifying the button's background color. + * If omitted, the default background color is used. + * @param {bool} [input.disabled] + * If `true`, the Touch Bar input is greyed out and inoperable. + * @param {Array} [input.children] + * An array of input objects that will be displayed as children of + * this input. Available only for types KInputTypes.POPOVER and + * kInputTypes.SCROLLVIEW. + */ + +Clarification on some of these properties is warranted. + +* ``title`` is the key to a Fluent translation defined in ``browser/locales/<LOCALE>/browser/touchbar/touchbar.ftl``. +* ``type`` must be a value from the ``kInputTypes`` enum in ``MacTouchBar.sys.mjs``. + For example, ``kInputTypes.BUTTON``. More information on input types follows + below. +* ``callback`` points to a JavaScript function. Any chrome-level JavaScript can + be executed. ``execCommand`` is a convenience method in ``MacTouchBar.sys.mjs`` + that takes a XUL command as a string and executes that command. For instance, + one input sets ``callback`` to ``execCommand("Browser:Back")``. +* ``children`` is an array of objects with the same properties as members of + ``gBuiltInInputs``. When used with an input of type + ``kInputTypes.SCROLLVIEW``, ``children`` can only contain inputs of type + ``kInputTypes.BUTTON``. When used with an input of type + ``kInputTypes.POPOVER``, any input type except another ``kInputTypes.POPOVER`` + can be used. + +.. _in that file: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/browser/components/touchbar/MacTouchBar.jsm#82 + +Input types +----------- + +Button + A simple button. If ``image`` is not specified, the buttons displays the text + label from ``title``. If both ``image`` and ``title`` are specified, only the + ``image`` is shown. The action specified in ``callback`` is executed when the + button is pressed. + + .. caution:: + + Even if the ``title`` will not be shown in the Touch Bar, you must still + define a ``title`` property. + +Main Button + Similar to a button, but displayed at double the width. A main button + displays both the string in ``title`` and the icon in ``image``. Only one + main button should be shown in the Touch Bar at any time, although this is + not enforced. + +Label + A non-interactive text label. This input takes only the attributes ``title`` + and ``type``. + +Popover + Initially represented in the Touch Bar as a button, a popover will display an + entirely different set of inputs when pressed. These different inputs should + be defined in the ``children`` property of the parent. Popovers can also be + shown and hidden programmatically, by calling + + .. highlight:: JavaScript + .. code:: + + gTouchBarUpdater.showPopover( + TouchBarHelper.baseWindow, + [POPOVER], + {true | false} + ); + + where the second argument is a reference to a popover TouchBarInput and + the third argument is whether the popover should be shown or hidden. + +Scroll View + A Scroll View is a scrolling list of buttons. The buttons should be defined + in the Scroll View's ``children`` array. + + .. note:: + + In Firefox, a list of search shortcuts appears in the Touch Bar when the + address bar is focused. This is an example of a ScrollView contained within + a popover. The popover is opened programmatically with + ``gTouchBarUpdater.showPopover`` when the address bar is focused and it is + hidden when the address bar is blurred. + +Examples +-------- +Some examples of ``gBuiltInInputs`` objects follow. + +A simple button + .. highlight:: JavaScript + .. code:: + + Back: { + title: "back", + image: "chrome://browser/skin/back.svg", + type: kInputTypes.BUTTON, + callback: () => execCommand("Browser:Back", "Back"), + }, + + A button is defined with a title, icon, type, and a callback. The callback + simply calls the XUL command to go back. + +The search popover + This is the input that occupies the Touch Bar when the address bar is focused. + + .. highlight:: JavaScript + .. code:: + + SearchPopover: { + title: "search-popover", + image: "chrome://global/skin/icons/search-glass.svg", + type: kInputTypes.POPOVER, + children: { + SearchScrollViewLabel: { + title: "search-search-in", + type: kInputTypes.LABEL, + }, + SearchScrollView: { + key: "search-scrollview", + type: kInputTypes.SCROLLVIEW, + children: { + Bookmarks: { + title: "search-bookmarks", + type: kInputTypes.BUTTON, + callback: () => + gTouchBarHelper.insertRestrictionInUrlbar( + UrlbarTokenizer.RESTRICT.BOOKMARK + ), + }, + History: { + title: "search-history", + type: kInputTypes.BUTTON, + callback: () => + gTouchBarHelper.insertRestrictionInUrlbar( + UrlbarTokenizer.RESTRICT.HISTORY + ), + }, + OpenTabs: { + title: "search-opentabs", + type: kInputTypes.BUTTON, + callback: () => + gTouchBarHelper.insertRestrictionInUrlbar( + UrlbarTokenizer.RESTRICT.OPENPAGE + ), + }, + Tags: { + title: "search-tags", + type: kInputTypes.BUTTON, + callback: () => + gTouchBarHelper.insertRestrictionInUrlbar( + UrlbarTokenizer.RESTRICT.TAG + ), + }, + Titles: { + title: "search-titles", + type: kInputTypes.BUTTON, + callback: () => + gTouchBarHelper.insertRestrictionInUrlbar( + UrlbarTokenizer.RESTRICT.TITLE + ), + }, + }, + }, + }, + }, + + At the top level, a Popover is defined. This allows a collection of children + to be shown in a separate Touch Bar. The Popover has two children: a Label, + and a Scroll View. The Scroll View displays five similar buttons that call a + helper method to insert search shortcut symbols into the address bar. + +Adding a new input +------------------ +Adding a new input is easy: just add a new object to ``gBuiltInInputs``. This +will make the input available in the Touch Bar customization window (accessible +from the Firefox menu bar item). + +If you want to to add your new input to the default set, add its identifier +here_, where ``type`` is a value from ``kAllowedInputTypes`` in that +file and ``key`` is the value you set for ``title`` in ``gBuiltInInputs``. +You should request approval from UX before changing the default set of inputs. + +.. _here: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/widget/cocoa/nsTouchBar.mm#100 + +If you are interested in adding new features to Firefox's implementation of the +Touch Bar API, read on! + + +Cocoa API +~~~~~~~~~ +Firefox implements Apple's Touch Bar API in its Widget: Cocoa code with an +``nsTouchBar`` class. ``nsTouchBar`` interfaces between Apple's Touch Bar API +and the ``TouchBarHelper`` JavaScript API. + +The best resource to understand the Touch Bar API is Apple's +`official documentation`_. This documentation will cover how Firefox implements +these APIs and how one might extend ``nsTouchBar`` to enable new Touch Bar +features. + +Every new Firefox window initializes ``nsTouchBar`` (link_). The function +``makeTouchBar`` is looked for automatically on every new instance of an +``NSWindow*``. If ``makeTouchBar`` is defined, that window will own a new +instance of ``nsTouchBar``. + +At the time of this writing, every window initializes ``nsTouchBar`` with a +default set of inputs. In the future, Firefox windows other than the main +browser window (such as the Library window or DevTools) may initialize +``nsTouchBar`` with a different set of inputs. + +``nsTouchBar`` has two different initialization methods: ``init`` and +``initWithInputs``. The former is a convenience method for the latter, calling +``initWithInputs`` with a nil argument. When that happens, a Touch Bar is +created containing a default set of inputs. ``initWithInputs`` can also take an +``NSArray<TouchBarInput*>*``. In that case, a non-customizable Touch Bar will be +initialized with only those inputs available. + +.. _official documentation: https://developer.apple.com/documentation/appkit/nstouchbar?language=objc +.. _link: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/widget/cocoa/nsCocoaWindow.mm#2877 + +NSTouchBarItemIdentifiers +------------------------- +The architecture of the Touch Bar is based largely around an ``NSString*`` +wrapper class called ``NSTouchBarItemIdentifier``. Every input in the Touch Bar +has a unique ``NSTouchBarItemIdentifier``. They are structured in reverse-URI +format like so: + +``com.mozilla.firefox.touchbar.[TYPE].[KEY]`` + +[TYPE] is a string indicating the type of the input, e.g. "button". If an +input is a child of another input, the parent's type is prepended to the child's +type, e.g. "scrubber.button" indicates a button contained in a scrubber. + +[KEY] is the ``title`` attribute defined for that input on the JS side. + +If you need to generate an identifier, use the convenience method +``[TouchBarInput nativeIdentifierWithType:withKey:]``. + +.. caution:: + + Do not create a new input that would have the same identifier as any other + input. All identifiers must be unique. + +.. warning:: + + ``NSTouchBarItemIdentifier`` `is used in one other place`_: setting + ``customizationIdentifier``. Do not ever change this string. If it is changed, + any customizations users have made to the layout of their Touch Bar in Firefox + will be erased. + +Each identifier is tied to a ``TouchBarInput``. ``TouchBarInput`` is a class +that holds the properties specified for each input in ``gBuiltInInputs``. +``nsTouchBar`` uses them to create instances of ``NSTouchBarItem`` +which are the actual objects used by Apple's Touch Bar API and displayed in the +Touch Bar. It is important to understand the difference between +``TouchBarInput`` and ``NSTouchBarItem``! + +.. _is used in one other place: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/widget/cocoa/nsTouchBar.mm#71 + +TouchBarInput creation flow +--------------------------- +Creating a Touch Bar and its ``TouchBarInputs`` flows as follows: + +#. ``[nsTouchBar init]`` is called from ``[NSWindow makeTouchBar]``. + +#. ``init`` populates two NSArrays: ``customizationAllowedItemIdentifiers`` and + ``defaultItemIdentifiers``. It also initializes a ``TouchBarInput`` object + for every element in the union of the two arrays and stores them in + ``NSMutableDictionary<NSTouchBarItemIdentifier, TouchBarInput*>* mappedLayoutItems``. + +#. ``touchBar:makeItemForIdentifier:`` is called for every element in the union + of the two arrays of identifiers. This method retrieves the ``TouchBarInput`` + for the given identifier and uses it to initialize a ``NSTouchBarItem``. + ``touchBar:makeItemForIdentifier:`` reads the ``type`` attribute from the + ``TouchBarInput`` to determine what ``NSTouchBarItem`` subclass should be + initialized. Our Touch Bar code currently supports ``NSCustomTouchBarItem`` + (buttons, main buttons); ``NSPopoverTouchBarItem`` (popovers); + ``NSTextField`` (labels); and ``NSScrollView`` (ScrollViews). + +#. Once the ``NSTouchBarItem`` is initialized, its properties are populated with + an assortment of "update" methods. These include ``updateButton``, + ``updateMainButton``, ``updateLabel``, ``updatePopover``, and + ``updateScrollView``. + +#. Since the localization of ``TouchBarInput`` titles happens asynchronously in + JavaScript code, the l10n callback executes + ``[nsTouchBarUpdater updateTouchBarInputs:]``. This method reads the + identifier of the input(s) that need to be updated and calls their respective + "update" methods. This method is most often used to update ``title`` after + l10n is complete. It can also be used to update any property of a + ``TouchBarInput``; for instance, one might wish to change ``color`` + when a specific event occurs in the browser. |