Touch Bar ========= The Touch Bar is a hardware component on some MacBook Pros released from 2016. It is a display above the keyboard that allows more flexible types of input than is otherwise possible with a normal keyboard. Apple offers Touch Bar APIs so developers can extend the Touch Bar to display inputs specific to their application. Firefox consumes these APIs to offer a customizable row of inputs in the Touch Bar. In Apple's documentation, the term "the Touch Bar" refers to the hardware. The term "a Touch Bar" refers not to the hardware but to a collection of inputs shown on the Touch Bar. This means that there can be multiple "Touch Bars" that switch out as the user switches contexts. The same naming convention is used in this document. In this document and in the code, the word "input" is used to refer to an interactive element in the Touch Bar. It is often interchangeable with "button", but "input" can also refer to any element displayed in the Touch Bar. The Touch Bar should never offer functionality unavailable to Firefox users without the Touch Bar. Most macOS Firefox users do not have the Touch Bar and some choose to disable it. Apple's own `Human Interface Guidelines`_ (HIG) forbids this kind of Touch Bar functionality. Please read the HIG for more design considerations before you plan on implementing a new Touch Bar feature. If you have questions about the Touch Bar that are not answered in this document, feel free to reach out to `Harry Twyford`_ (:harry on Slack). He wrote this document and Firefox's initial Touch Bar implementation. .. _Human Interface Guidelines: https://developer.apple.com/design/human-interface-guidelines/macos/touch-bar/touch-bar-overview/ .. _Harry Twyford: mailto:harry@mozilla.com .. contents:: Table of Contents Overview ~~~~~~~~ Firefox's Touch Bar implementation is equal parts JavaScript and Cocoa (Objective-C++). The JavaScript code lives in ``browser/components/touchbar`` and the Cocoa code lives in ``widget/cocoa``, mostly in ``nsTouchBar.mm``. The Cocoa code is a consumer of Apple's Touch Bar APIs and defines what types of Touch Bar inputs are available to its own consumers. The JS code in ``browser/components/touchbar`` provides services to ``nsTouchBar.mm`` and defines what inputs the user actually sees in the Touch Bar. There is two-way communication between the JS and the Cocoa: the Cocoa code asks the JS what inputs it should display, and the JS asks the Cocoa code to update those inputs when needed. JavaScript API ~~~~~~~~~~~~~~ ``browser/components/touchbar/MacTouchBar.sys.mjs`` defines what specific inputs are available to the user, what icon they will have, what action they will perform, and so on. Inputs are defined in the ``gBuiltInInputs`` object `in that file`_. When creating a new object in ``gBuiltInInputs``, the available properties are documented in the JSDoc for ``TouchBarInput``: .. highlight:: JavaScript .. code:: /** * A representation of a Touch Bar input. * @param {string} input.title * The lookup key for the button's localized text title. * @param {string} input.image * A URL pointing to an SVG internal to Firefox. * @param {string} input.type * The type of Touch Bar input represented by the object. * Must be a value from kInputTypes. * @param {Function} input.callback * A callback invoked when a touchbar item is touched. * @param {string} [input.color] * A string in hex format specifying the button's background color. * If omitted, the default background color is used. * @param {bool} [input.disabled] * If `true`, the Touch Bar input is greyed out and inoperable. * @param {Array} [input.children] * An array of input objects that will be displayed as children of * this input. Available only for types KInputTypes.POPOVER and * kInputTypes.SCROLLVIEW. */ Clarification on some of these properties is warranted. * ``title`` is the key to a Fluent translation defined in ``browser/locales//browser/touchbar/touchbar.ftl``. * ``type`` must be a value from the ``kInputTypes`` enum in ``MacTouchBar.sys.mjs``. For example, ``kInputTypes.BUTTON``. More information on input types follows below. * ``callback`` points to a JavaScript function. Any chrome-level JavaScript can be executed. ``execCommand`` is a convenience method in ``MacTouchBar.sys.mjs`` that takes a XUL command as a string and executes that command. For instance, one input sets ``callback`` to ``execCommand("Browser:Back")``. * ``children`` is an array of objects with the same properties as members of ``gBuiltInInputs``. When used with an input of type ``kInputTypes.SCROLLVIEW``, ``children`` can only contain inputs of type ``kInputTypes.BUTTON``. When used with an input of type ``kInputTypes.POPOVER``, any input type except another ``kInputTypes.POPOVER`` can be used. .. _in that file: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/browser/components/touchbar/MacTouchBar.jsm#82 Input types ----------- Button A simple button. If ``image`` is not specified, the buttons displays the text label from ``title``. If both ``image`` and ``title`` are specified, only the ``image`` is shown. The action specified in ``callback`` is executed when the button is pressed. .. caution:: Even if the ``title`` will not be shown in the Touch Bar, you must still define a ``title`` property. Main Button Similar to a button, but displayed at double the width. A main button displays both the string in ``title`` and the icon in ``image``. Only one main button should be shown in the Touch Bar at any time, although this is not enforced. Label A non-interactive text label. This input takes only the attributes ``title`` and ``type``. Popover Initially represented in the Touch Bar as a button, a popover will display an entirely different set of inputs when pressed. These different inputs should be defined in the ``children`` property of the parent. Popovers can also be shown and hidden programmatically, by calling .. highlight:: JavaScript .. code:: gTouchBarUpdater.showPopover( TouchBarHelper.baseWindow, [POPOVER], {true | false} ); where the second argument is a reference to a popover TouchBarInput and the third argument is whether the popover should be shown or hidden. Scroll View A Scroll View is a scrolling list of buttons. The buttons should be defined in the Scroll View's ``children`` array. .. note:: In Firefox, a list of search shortcuts appears in the Touch Bar when the address bar is focused. This is an example of a ScrollView contained within a popover. The popover is opened programmatically with ``gTouchBarUpdater.showPopover`` when the address bar is focused and it is hidden when the address bar is blurred. Examples -------- Some examples of ``gBuiltInInputs`` objects follow. A simple button .. highlight:: JavaScript .. code:: Back: { title: "back", image: "chrome://browser/skin/back.svg", type: kInputTypes.BUTTON, callback: () => execCommand("Browser:Back", "Back"), }, A button is defined with a title, icon, type, and a callback. The callback simply calls the XUL command to go back. The search popover This is the input that occupies the Touch Bar when the address bar is focused. .. highlight:: JavaScript .. code:: SearchPopover: { title: "search-popover", image: "chrome://global/skin/icons/search-glass.svg", type: kInputTypes.POPOVER, children: { SearchScrollViewLabel: { title: "search-search-in", type: kInputTypes.LABEL, }, SearchScrollView: { key: "search-scrollview", type: kInputTypes.SCROLLVIEW, children: { Bookmarks: { title: "search-bookmarks", type: kInputTypes.BUTTON, callback: () => gTouchBarHelper.insertRestrictionInUrlbar( UrlbarTokenizer.RESTRICT.BOOKMARK ), }, History: { title: "search-history", type: kInputTypes.BUTTON, callback: () => gTouchBarHelper.insertRestrictionInUrlbar( UrlbarTokenizer.RESTRICT.HISTORY ), }, OpenTabs: { title: "search-opentabs", type: kInputTypes.BUTTON, callback: () => gTouchBarHelper.insertRestrictionInUrlbar( UrlbarTokenizer.RESTRICT.OPENPAGE ), }, Tags: { title: "search-tags", type: kInputTypes.BUTTON, callback: () => gTouchBarHelper.insertRestrictionInUrlbar( UrlbarTokenizer.RESTRICT.TAG ), }, Titles: { title: "search-titles", type: kInputTypes.BUTTON, callback: () => gTouchBarHelper.insertRestrictionInUrlbar( UrlbarTokenizer.RESTRICT.TITLE ), }, }, }, }, }, At the top level, a Popover is defined. This allows a collection of children to be shown in a separate Touch Bar. The Popover has two children: a Label, and a Scroll View. The Scroll View displays five similar buttons that call a helper method to insert search shortcut symbols into the address bar. Adding a new input ------------------ Adding a new input is easy: just add a new object to ``gBuiltInInputs``. This will make the input available in the Touch Bar customization window (accessible from the Firefox menu bar item). If you want to to add your new input to the default set, add its identifier here_, where ``type`` is a value from ``kAllowedInputTypes`` in that file and ``key`` is the value you set for ``title`` in ``gBuiltInInputs``. You should request approval from UX before changing the default set of inputs. .. _here: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/widget/cocoa/nsTouchBar.mm#100 If you are interested in adding new features to Firefox's implementation of the Touch Bar API, read on! Cocoa API ~~~~~~~~~ Firefox implements Apple's Touch Bar API in its Widget: Cocoa code with an ``nsTouchBar`` class. ``nsTouchBar`` interfaces between Apple's Touch Bar API and the ``TouchBarHelper`` JavaScript API. The best resource to understand the Touch Bar API is Apple's `official documentation`_. This documentation will cover how Firefox implements these APIs and how one might extend ``nsTouchBar`` to enable new Touch Bar features. Every new Firefox window initializes ``nsTouchBar`` (link_). The function ``makeTouchBar`` is looked for automatically on every new instance of an ``NSWindow*``. If ``makeTouchBar`` is defined, that window will own a new instance of ``nsTouchBar``. At the time of this writing, every window initializes ``nsTouchBar`` with a default set of inputs. In the future, Firefox windows other than the main browser window (such as the Library window or DevTools) may initialize ``nsTouchBar`` with a different set of inputs. ``nsTouchBar`` has two different initialization methods: ``init`` and ``initWithInputs``. The former is a convenience method for the latter, calling ``initWithInputs`` with a nil argument. When that happens, a Touch Bar is created containing a default set of inputs. ``initWithInputs`` can also take an ``NSArray*``. In that case, a non-customizable Touch Bar will be initialized with only those inputs available. .. _official documentation: https://developer.apple.com/documentation/appkit/nstouchbar?language=objc .. _link: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/widget/cocoa/nsCocoaWindow.mm#2877 NSTouchBarItemIdentifiers ------------------------- The architecture of the Touch Bar is based largely around an ``NSString*`` wrapper class called ``NSTouchBarItemIdentifier``. Every input in the Touch Bar has a unique ``NSTouchBarItemIdentifier``. They are structured in reverse-URI format like so: ``com.mozilla.firefox.touchbar.[TYPE].[KEY]`` [TYPE] is a string indicating the type of the input, e.g. "button". If an input is a child of another input, the parent's type is prepended to the child's type, e.g. "scrubber.button" indicates a button contained in a scrubber. [KEY] is the ``title`` attribute defined for that input on the JS side. If you need to generate an identifier, use the convenience method ``[TouchBarInput nativeIdentifierWithType:withKey:]``. .. caution:: Do not create a new input that would have the same identifier as any other input. All identifiers must be unique. .. warning:: ``NSTouchBarItemIdentifier`` `is used in one other place`_: setting ``customizationIdentifier``. Do not ever change this string. If it is changed, any customizations users have made to the layout of their Touch Bar in Firefox will be erased. Each identifier is tied to a ``TouchBarInput``. ``TouchBarInput`` is a class that holds the properties specified for each input in ``gBuiltInInputs``. ``nsTouchBar`` uses them to create instances of ``NSTouchBarItem`` which are the actual objects used by Apple's Touch Bar API and displayed in the Touch Bar. It is important to understand the difference between ``TouchBarInput`` and ``NSTouchBarItem``! .. _is used in one other place: https://searchfox.org/mozilla-central/rev/ebe492edacc75bb122a2b380e4cafcca3470864c/widget/cocoa/nsTouchBar.mm#71 TouchBarInput creation flow --------------------------- Creating a Touch Bar and its ``TouchBarInputs`` flows as follows: #. ``[nsTouchBar init]`` is called from ``[NSWindow makeTouchBar]``. #. ``init`` populates two NSArrays: ``customizationAllowedItemIdentifiers`` and ``defaultItemIdentifiers``. It also initializes a ``TouchBarInput`` object for every element in the union of the two arrays and stores them in ``NSMutableDictionary* mappedLayoutItems``. #. ``touchBar:makeItemForIdentifier:`` is called for every element in the union of the two arrays of identifiers. This method retrieves the ``TouchBarInput`` for the given identifier and uses it to initialize a ``NSTouchBarItem``. ``touchBar:makeItemForIdentifier:`` reads the ``type`` attribute from the ``TouchBarInput`` to determine what ``NSTouchBarItem`` subclass should be initialized. Our Touch Bar code currently supports ``NSCustomTouchBarItem`` (buttons, main buttons); ``NSPopoverTouchBarItem`` (popovers); ``NSTextField`` (labels); and ``NSScrollView`` (ScrollViews). #. Once the ``NSTouchBarItem`` is initialized, its properties are populated with an assortment of "update" methods. These include ``updateButton``, ``updateMainButton``, ``updateLabel``, ``updatePopover``, and ``updateScrollView``. #. Since the localization of ``TouchBarInput`` titles happens asynchronously in JavaScript code, the l10n callback executes ``[nsTouchBarUpdater updateTouchBarInputs:]``. This method reads the identifier of the input(s) that need to be updated and calls their respective "update" methods. This method is most often used to update ``title`` after l10n is complete. It can also be used to update any property of a ``TouchBarInput``; for instance, one might wish to change ``color`` when a specific event occurs in the browser.