diff options
Diffstat (limited to 'iphone/doc/optimizing.rst')
| -rw-r--r-- | iphone/doc/optimizing.rst | 435 |
1 files changed, 435 insertions, 0 deletions
diff --git a/iphone/doc/optimizing.rst b/iphone/doc/optimizing.rst new file mode 100644 index 0000000..a823fd6 --- /dev/null +++ b/iphone/doc/optimizing.rst @@ -0,0 +1,435 @@ +Optimizing the Reader +===================== + +As good as the iPhone performance is for a mobile device, the reality from an +image processing perspective is that it represents a lower performance target. +While the default configuration of the iPhone reader has been carefully tuned +for the general case, you can often obtain much better results if you optimize +for your specific application. + +.. note:: + + Performance is a complex topic. The only way to tune performance is by + changing settings and comparing measured results. If you are not + comfortable with the concepts presented here, it is recommended that you + leave the settings at the defaults and avoid compromising reliability. + +Performance of the barcode reader is generally evaluated by two factors: + +* The **latency** - how quickly it "recognizes" a barcode. Specifically this + is the time from when the user puts a barcode in the frame or selects an + image until a response is indicated back to the them. + +* The **reliability** - it does not matter how quickly an image is scanned if + an incorrect result is returned. That may seem obvious, but bad decodes + *are* possible and you need to keep this in mind when changing settings that + affect the reliability of the reader. + +Basically our goal is to optimize the latency without sacrificing reliability. +There are several factors that contribue to latency: + +* The **quality** of the barcode image. Quality includes the available + resolution, focus, lighting, noise, etc. We have more control over some of + these than others. + +* The **camera**. When scanning from the camera, the time for the + autoexposure and autofocus to converge on an image that can be decoded is a + significant contribution to the overall latency. + +* The **frame rate** of the reader - this translates to the time it takes the + scanner to process an image. + +* The **effort level** of the reader - some of the available settings control + "how hard" the reader tries to find barcodes in each frame. + +* The **delegate latency** - the time spent in your application after a + barcode has been detected until the user is notified. + +Most of these factors are interrelated. We will discuss those we can control +in detail, as well the settings you use to affect them. Then we will provide +a few specific case examples. + + +Measuring Performance +--------------------- + +Subjective response times are a good place to start (does it "feel" responsive +to you?), and possibly the only way to evaluate the overall experience, but to +compare incremental changes to interrelated settings and have meaningful +performance discussions with others, we need a more quantitative approach. + +The :func:`mach_absolute_time` function is a good tool for accurately +measuring small delays. Research this function and learn how to apply it. As +when measuring any real-world value, keep in mind that some variance is to be +expected - even if you perform exactly the same operation multiple times, you +will not see exactly the same measurement. You should collect several +samples, discard any obvious outliers, and average the remaining measurements. + +One way that the overall reader latency may be evaluated is by manually +marking the time when the barcode is presented to the reader. Add a control +to your overlay that captures the start time when tapped and compare this to +the end time, just before your delegate returns. + +The reader continually monitors the frame rate at which it is running. The +measured value may be displayed for debugging purposes by enabling the +:member:`~ZBarReaderView::showsFPS` property. The readertest example does +this and also provides control over many of the available settings, so you can +quickly test how each setting affects the frame rate. You should target your +optimization efforts to achieve a frame rate of at least 8-10fps, although +12-15fps is preferable. + +You can measure the latency of your delegate using :func:`mach_absolute_time`. +The measured value should be less than about 100ms, the smaller the better, to +avoid noticeable lag. + +The readertest is a good tool for testing the performance of the reader. You +can tune the settings appropriately for your application and evaluate the +effect each change has on the performance. + + +Delegate Latency +---------------- + +This latency contributor is the easiest for you to effect (and sometimes the +easiest to overlook). Your delegate method should update the interface - +dismiss the controller or update your overlay to indicate success - and +*nothing* else. All other processing should be deferred until after the +animations have started. + + +Image Quality +------------- + +Resolution +^^^^^^^^^^ + +One might think that "more is better" in terms of resolution, but this is not +necessarily the case. Given average image quality, the ideal resolution for +scanning is right around three pixels per barcode "module" (the width of the +smallest bar or space). Note that this measure is not an absolute image size +or even a measure of the physical dimensions represented by a pixel sample, it +*only* describes the sampled size of the barcode in the image. + +As the resolution decreases below about two pixels per module, edge fidelity +is lost and the bars and spaces start to merge together, making it impossible +(for this library) to scan. This affects the density (feature size) and +maximum size (data capacity) of the barcodes that can be detected. +Conversely, as the resolution increases above about 4 pixels per module, noise +can interfere with the edge detection and images will take longer to process. + +Other quality factors, such as poor focus, bad lighting or even excessive +noise, can increase (or decrease) the resolution requirement. + +When scanning from the camera, the reader defaults to 640x480, which is good +for most applications. On newer devices, you can increase this using a capture +:member:`~ZBarReaderView::session` preset. Some older devices do not have a +higher resolution option available. + +For scanning images, you can use +:member:`~ZBarReaderController::maxScanDimension` to control the scaled size +of the converted image, or resort to converting them yourself. + +If you want to read long linear barcodes or dense 2-D symbols, you will +probably want to increase the resolution by adjusting these settings. + +Keep in mind that more pixels will take longer to scan, refer to the `frame +rate`_ discussion for ways to compensate. + +Focus +^^^^^ + +Ideally we would fix the focus at a calculated optimum distance and optimize +the aperture selection to maximize the depth of field. Unfortunately the APIs +do not currently give us control over any of these settings, the best we can +do (as of iOS 4) is continuous auto-focus mode - this mode is configured by +the reader automatically. It can still take the device as long as 1-2 seconds +to find the appropriate macro focus setting, but again, there is currently no +way to reduce this delay. + +Lighting and Exposure +^^^^^^^^^^^^^^^^^^^^^ + +An image that is too bright or overexposed can completely wash out any +barcodes. An image that is too dark or underexposed will not provide +sufficient contrast for the scanner. Low light levels also tend to produce +noisier images, possibly because the driver uses a faster "ISO" setting to +compensate for the lighting. + +The camera defaults to continuous automatic exposure and white balance. Since +there are no other useful values, the reader leaves these unchanged from their +default setting. + +For some devices, the "torch" can be enabled to provide additional +illumination for the camera in low-light conditions. The reader sets the +torch to automatic by default, so it should turn on only when needed... +There have been some reports that the torch turns on inappropriately, washing +out the image. If you find that this occurs, you should instead set the +:member:`~ZBarReaderView::torchMode` property of the :class:`ZBarReaderView` +to ``Off``. + +For scanning images from another source, you are again stuck with the +available image quality. If you have any control over the image source, you +should do what you can to fix quality problems there. + +Noise +^^^^^ + +Some level of noise is filtered by the reader, but excessive noise levels +create additional edges in the image which corrupt barcodes and increase +scanning time (decreasing the frame rate). + +As mentioned with `lighting and exposure`_, noise mostly becomes a problem +when the light-level is too low, but high-resolution images may also increase +exposure to sensor noise. + +We compensate for noise by *reducing* the `resolution`_ from the sensor +maximum. Scaling the image down has the effect of averaging several pixels +into one value, filtering out the high-frequency noise component. + + +Frame Rate +---------- + +The time it takes to scan and decode an image/frame is roughly proportional to +the number of pixels that are processed. The number and type of enabled +symbologies and image noise can also affect the processing time. + +We have several knobs available that affect the frame rate. Most of these are +geared toward reducing the number of image pixels that are scanned. + +Decrease the Resolution +^^^^^^^^^^^^^^^^^^^^^^^ + +Adjusting the resolution of the image is an easy way to quickly reduce the +number of pixels. Smaller images also mean there is less data to carry +around, which helps performance in other ways. For example, reducing each +image dimension by 30% (eg, from 640x480 to 448x336) will about double the +speed of the reader (to a point). [FIXME verify!] + +Adjusting the resolution is `described above <resolution>`_. As mentioned +there, reducing the resolution will negatively impact the minimum feature size +and maximum barcode size that can be scanned, but it will help filter noise. + +Crop the Scan Region +^^^^^^^^^^^^^^^^^^^^ + +It may not always be necessary for an application to scan all the way to the +edges of the image. By cropping the scan area, you can get most of the +benefits of reduced resolution without sacrificing the minimum feature size. +Cropping will also not affect image noise, but similar to decreasing the +resolution, it does affect the maximum size barcode that can be scanned. + +For all cases you set the crop rectangle +:class:`~ZBarReaderViewController::scanCrop` property. Note that the +rectangle provided to the controller is *normalized* across image size and +rotation. This means that the coordinates range from 0 to 1 and the axes will +be arranged such that the x-axis of the crop rectangle corresponds to the +major (longer) image axis. + +Your interface will typically need to indicate the cropped scan area to the +user with visual queues. Use the +:class:`~ZBarReaderViewController::cameraOverlayView` to provide this. + +By default, the :class:`ZBarReaderView` recognizes a pinch gesture to +digitally zoom the preview around the center of the image. This zoom does not +affect the resolution of the image, but it does crop the scan region to the +visible area. You can also disable the pinch gesture and set the +:class:`~ZBarReaderView::zoom` programmatically. + +Limit the Scan Density +^^^^^^^^^^^^^^^^^^^^^^ + +The scanner works by making scan passes across the pixel rows and colums of +the image. The density of the passes is configured at the scanner as a pixel +stride for each axis. ``ZBAR_CFG_Y_DENSITY`` (``ZBAR_CFG_X_DENSITY``) +controls the number of pixel rows (columns) that are skipped between +successive horizontal (vertical) scan passes. (Note that "density" is really +not a good name for the configuration settings... "stride" might be more +appropriate.) + +Decreasing the scan density (by increasing the stride setting) is a great way +to limit the processing (increasing the frame rate) without sacrificing scan +resolution - each scan pass is still made at full image resolution, there are +just fewer passes (less redundancy). + +Setting the stride value to 0 completely disables scanning in that direction. +This is very useful when reading linear codes with a visual alignment guide - +scanning parallel to the bars is a waste of cycles which may be better applied +to support higher resolution or increased density of scans across the symbol. +Note that some 2-D symbologies (QR Code) require scans in both directions. + +Setting the stride to a very large value will generate a single scan pass +through the center of the image. Note that some symbologies will not be +detected without multiple successful passes; it is usually better to combine +this setting with cropping to generate a number of closely clustered scan +passes in the target area. + +Note that the density also affects the aspect ratio and rotation that can be +tolerated. If you set it too large, some barcodes will become more difficult +to read. + +In general, 2 to 4 is a good target for the stride setting, unless you have +very high or low resolution images. + +Disable unused symbologies +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Limiting the symbologies to the set of interest should provide a small +performance boost. It also improves decode reliability - it is impossible to +receive an incorrect or unexpected decode result from a symbology that is +disabled. + +The reader does support full auto-discrimination among the supported +symbologies, but with all of them enabled you may need to compensate elsewhere +to get a good frame rate. + +For example, if you are only interested in QR codes, disable the others. The +robust way to do this is by disabling *all* symbologies and then reenabling +only those you want. This helps isolate you from encountering new symbologies +that may be added in future versions of the library until you are ready to +handle them:: + + [scanner setSymbology: 0 + config: ZBAR_CFG_ENABLE + to: 0]; + [scanner setSymbology: ZBAR_QRCODE + config: ZBAR_CFG_ENABLE + to: 1]; + +Even if you would like your application to support multiple symbologies, you +may consider if there is a way to limit the enabled subset based on the +scanning context, etc... + + +Examples +-------- + +These examples demonstrate several scenarios for scanning from the camera with +automatic capture. You can try them yourself using the readertest. For each +example, start with the default settings (by tapping the +``ZBarReaderViewController`` class), then enable continuous mode and the +custom overlay (by disabling +:member:`~ZBarReaderViewController::showsZBarControls`). You should also use +a release build and avoid running in the debugger. + +Frame rates are approximate, measured on an iPhone 3GS running iOS 4.0.1 in a +well lit room. Two measurements are taken for each sample: the rate with the +camera pointed at a blank white page such that it fills the frame, and the +rate while continuously decoding the provided example. For best results, it +is recommended that you print the examples rather than scanning them from the +screen. + +For reference, the base frame rates with default settings are 12fps for a +blank white page, 7.5fps for this `basic EAN symbol`_ and 2.2fps for this +`basic QR symbol`_. + +.. _`basic EAN symbol`: + http://zbar.sf.net/test/ean13/9876543210128.png +.. _`basic QR symbol`: + http://chart.apis.google.com/chart?cht=qr&chs=512x512&chl=http://zbar.sf.net/iphone + +Long Linear Symbols +^^^^^^^^^^^^^^^^^^^ + +For this example, we will use a relatively `long Code 128 barcode`_. + +.. _`long Code 128 barcode`: + http://zbar.sf.net/test/code128/ALPHA.png + +While it should be possible to read this symbol with the default settings, you +may notice that it is not very reliable. You will have to stretch the symbol +across the entire screen, and even then the default settings will only give +you about 1.6 pixels per module, well below the ideal target of 3. To improve +these results, we want to maximize scanning resolution for the long image +axis. + +1. Disable the default zoom/crop - zoom all the way out by hitting "Scan" and + pinching the preview; the frame rate immediately drops to 8fps / 4.8fps. + +We should compensate for this reduction in the frame rate: + +2. Crop the image to a long, skinny rectangle - set the + :member:`~ZBarReaderViewController::scanCrop` setting to + ``{{0, 0.3}, {1, 0.4}}``; The frame rate jumps up to 18fps / 8.7fps. + +3. Disable scans across the short image axis - set the ``CFG_X_DENSITY`` + setting to 0. The frame rate goes all the way to 30fps / 13fps. + +Since we have plenty of margin with the frame rate, we can minimize the total +decode latency by performing more scan passes through the symbol: + +4. Increase the scan density - set the ``CFG_Y_DENSITY`` setting to 1 (13.5fps + / 5fps) or 2 (24fps / 9fps). + +You should now be able to quickly and reliably decode long linear symbols. + +If have a newer device, you may also try increasing the resolution to support +even longer symbols. You may have to compensate elsewhere to bring the frame +rate back to a reasonable level. + +High Density QR Symbols +^^^^^^^^^^^^^^^^^^^^^^^ + +For this example we will use a `version 29 QR Code symbol`_. + +.. _`version 29 QR Code symbol`: + http://www.qrcomic.com/images/5.png + +In this case we still want to maximize the resolution, but we also need to +increase the scan density to reliably pick up the small finder patterns: + +1. Maximize scan density in both directions - set the ``CFG_X_DENSITY`` and + ``CFG_Y_DENSITY`` settings both to 1. You should be able to scan the symbol + now, although the frame rate drops to 4.5fps / 1fps + +2. Disable the default zoom/crop - zoom all the way out by hitting "Scan" and + pinching the preview; the frame rate drops further to 3fps / 0.7fps + +We can compensate somewhat for the reduced frame rate: + +3. Crop the image to a square - set ``scanCrop`` to ``{{0.125, 0}, {.75, 1}}``. + This boosts the frame rate slightly to 3.7fps / 0.75fps. + +4. Disable linear symbologies - set the symbologies such that only QR Code is + enabled (4fps / 1fps) + +Even though the frame rate is still pretty bad, the QR recognition latency +should be acceptable. + +If have an iPhone 4, you may also try increasing the resolution to support +even denser QR symbols. You may have to compensate elsewhere to bring the +frame rate back to a reasonable level. + +Small DataBar Symbols +^^^^^^^^^^^^^^^^^^^^^ + +For this example we will use a `DataBar symbol`_ printed with a small feature +size, typical of the stickers used to tag produce. Scale it when printing +such that the printed dimensions are about 1cm square. This symbol should +scan with the default settings, but we will attempt to optimize the scan +latency for this case. + +.. _`DataBar symbol`: + http://zbar.sf.net/test/databar/0109876543210128-so.png + +As well as high barcode resolution, we also want high density passes in both +directions to minimize sensitivity to rotation: + +1. Maximize scan density in both directions - set the ``CFG_X_DENSITY`` and + ``CFG_Y_DENSITY`` settings both to 1. The frame rate drops to 4.5fps / + 3fps. + +Compensate for the reduction in frame rate by zooming in on the small symbol, +which crops the scanned image. Zooming also helps the user see the small +barcode: + +2. Zoom all the way in - hit "Scan" and un-pinch the preview. The frame rate + recovers to 11fps / 6.2fps. + +3. Crop the image to a square - set ``scanCrop`` to ``{{0.125, 0}, {0.75, 1}}`` + (14fps / 7.5fps) + +4. Disable all symbologies except DataBar and DataBar Expanded (14.5fps / 9fps) + +The reader should now be very sensitive to DataBar, even when scanned at an +angle. |