Here, po4a searches for the first line matching C<<

>> after the line matching C in the translation, and add the addendum B that line since we provided a I, i.e. a boundary marking the beginning of the next section. So this header line requires placing the addendum after the section containing C, and instruct po4a that a section starts with a line containing the C<<

>> tag. This is equivalent to the previous example because what you really want is to add this addendum either after C<<

>> or before C<<

>>. You can also set the insertion I to the value C, with a similar semantic: combining C with an C will put the addendum just B the matched boundary, that is the last potential boundary line before the C. Combining C with an C will put the addendum just B the matched boundary, that is the last potential boundary line before the C. Mode | Boundary kind | Used boundary | Insertion point compared to the boundary ========|===============|========================|========================================= 'before'| 'endboundary' | last before 'position' | Right after the selected boundary 'before'|'beginboundary'| last before 'position' | Right before the selected boundary 'after' | 'endboundary' | first after 'position' | Right after the selected boundary 'after' |'beginboundary'| first after 'position' | Right before the selected boundary 'eof' | (none) | n/a | End of file =head3 Hint and tricks about addenda =over =item Remember that these are regexp. For example, if you want to match the end of a nroff section ending with the line C<.fi>, do not use C<.fi> as B, because it will match with C, which is obviously not what you expect. The correct B in that case is: C<^\.fi$>. =item White spaces ARE important in the content of the C and boundaries. So the two following lines B. The second one will only be found if there is enough trailing spaces in the translated document. PO4A-HEADER: position=About this document; mode=after; beginboundary=

PO4A-HEADER: position=About this document ; mode=after; beginboundary=

=item Although this context search may be considered to operate roughly on each line of the B document, it actually operates on the internal data string of the translated document. This internal data string may be a text spanning a paragraph containing multiple lines or may be a XML tag itself alone. The exact I of the addendum must be before or after the internal data string and can not be within the internal data string. =item Pass the C<-vv> argument to B to understand how the addenda are added to the translation. It may also help to run B in debug mode to see the actual internal data string when your addendum does not apply. =back =head3 Addenda examples =over =item If you want to add something after the following nroff section: .SH "AUTHORS" You should select a two-step approach by setting B. Then you should narrow down search to the line after B with the B argument regex. Then, you should match the beginning of the next section (i.e., B<^\.SH>) with the B argument regex. That is to say: PO4A-HEADER:mode=after;position=AUTHORS;beginboundary=\.SH =item If you want to add something right after a given line (e.g. after the line "Copyright Big Dude"), use a B matching this line, B and give a B matching any line. PO4A-HEADER:mode=after;position=Copyright Big Dude, 2004;beginboundary=^ =item If you want to add something at the end of the document, give a B matching any line of your document (but only one line. Po4a won't proceed if it's not unique), and give an B matching nothing. Don't use simple strings here like B<"EOF">, but prefer those which have less chance to be in your document. PO4A-HEADER:mode=after;position=About this document;beginboundary=FakePo4aBoundary =back =head3 More detailed example Original document (POD formatted): |=head1 NAME | |dummy - a dummy program | |=head1 AUTHOR | |me Then, the following addendum will ensure that a section (in French) about the translator is added at the end of the file (in French, "TRADUCTEUR" means "TRANSLATOR", and "moi" means "me"). |PO4A-HEADER:mode=after;position=AUTEUR;beginboundary=^=head | |=head1 TRADUCTEUR | |moi | To put your addendum before the AUTHOR, use the following header: PO4A-HEADER:mode=after;position=NOM;beginboundary=^=head1 This works because the next line matching the B C after the section "NAME" (translated to "NOM" in French), is the one declaring the authors. So, the addendum will be put between both sections. Note that if another section is added between NAME and AUTHOR sections later, po4a will wrongfully put the addenda before the new section. To avoid this you may accomplish the same using B=I: PO4A-HEADER:mode=before;position=^=head1 AUTEUR =head1 How does it work? This chapter gives you a brief overview of the po4a internals, so that you may feel more confident to help us to maintain and to improve it. It may also help you to understand why it does not do what you expected, and how to solve your problems. =head2 TransTractors and project architecture At the core of the po4a project, the L class is the common ancestor to all po4a parsers. This strange name comes from the fact that it is at the same time in charge of translating document and extracting strings. More formally, it takes a document to translate plus a PO file containing the translations to use as input while producing two separate outputs: Another PO file (resulting of the extraction of translatable strings from the input document), and a translated document (with the same structure as the input one, but with all translatable strings replaced with content of the input PO). Here is a graphical representation of this: Input document --\ /---> Output document \ TransTractor:: / (translated) +-->-- parse() --------+ / \ Input PO --------/ \---> Output PO (extracted) This little bone is the core of all the po4a architecture. If you provide both input and disregard the output PO, you get B. If you disregard the output document instead, you get B. The B uses a first TransTractor to get an up-to-date output POT file (disregarding the output documents), calls B to update the translation PO files on disk, and builds a second TransTractor with these updated PO files to update the output documents. In short, B provides one-stop solution to update what needs to be, using a single configuration file. B also uses two TransTractors, but another way: It builds one TransTractor per language, and then build a new PO file using the msgids of the original document as msgids, and the msgids of the translated document as msgstrs. Much care is needed to ensure that the strings matched this way actually match, as described in L. =head2 Format-specific parsers All po4a format parsers are implemented on top of the TransTractor. Some of them are very simple, such as the Text, Markdown and AsciiDoc ones. They load the lines one by one using C, accumulate the paragraphs' content or whatever. Once a string is completely parsed, the parser uses C to (1) add this string to the output PO file and (2) get the translation from the input PO file. The parser then pushes the result to the output file using C. Some other parsers are more complex because they rely on an external parser to analyze the input document. The Xml, HTML, SGML and Pod parsers are built on top of SAX parsers. They declare callbacks to events such as "I found a new title which content is the following" to update the output document and output POT files according to the input content using C and C. The Yaml parser is similar but different: it serializes a data structure produced by the YAML::Tiny parser. This is why the Yaml module of po4a fails to declare the reference lines: the location of each string in the input file is not kept by the parser, so we can only provide "$filename:1" as a string location. The SAX-oriented parsers use globals and other tricks to save the file name and line numbers of references. One specific issue arises from file encodings and BOM markers. Simple parsers can forget about this issue, that is handled by C (used internally to get the lines of an input document), but the modules relying on an external parser must ensure that all files are read with an appropriate PerlIO decoding layer. The easiest is to open the file yourself, and provide an filehandle or directly the full string to your external parser. Check on C and C for an example. The content read by the TransTractor is ignored, but a fresh filehandle is passed to the external parser. The important part is the C<< "<:encoding($charset)" >> mode that is passed to the B perl function. =head2 Po objects The L class is in charge of loading and using PO and POT files. Basically, you can read a file, add entries, get translations with the B method, write the PO into a file. More advanced features such as merging a PO file against a POT file or validating a file are delegated to B and B respectively. =head2 Contributing to po4a Even if you have never contributed to any Open Source project in the past, you are welcome: we are willing to help and mentor you here. po4a is best maintained by its users nowadays. As we lack manpower, we try to make the project welcoming by improving the doc and the automatic tests to make you confident in contributing to the project. Please refer to the CONTRIBUTING.md file for more details. =head1 Open-source projects using po4a Here is a very partial list of projects that use po4a in production for their documentation. If you want to add your project to the list, just drop us an email (or a Merge Request). =over =item adduser (man): users and groups management tool. =item apt (man, docbook): Debian package manager. =item aptitude (docbook, svg): terminal-based package manager for Debian =item L (markdown): installable catalog of FOSS (Free and Open Source Software) applications for the Android platform. =item L (asciidoc): distributed version-control system for tracking changes in source code. =item L (man) This project provides an infrastructure for translating many manpages to different languages, ready for integration into several major distributions (Arch Linux, Debian and derivatives, Fedora). =item L (HTML): a free open source planetarium for your computer. po4a is used to translate the sky culture descriptions. =item L (markdown, yaml, HTML): a FOSS application for online jamming in real time. The website documentation is maintained in multiple languages using po4a. =item Other item to sort out: L L =back =head1 FAQ =head2 How do you pronounce po4a? I personally vocalize it as L, which is a French onomatopoetic that we use in place of yuck :) I may have a strange sense of humor :) =head2 Why are the individual scripts deprecated? Indeed, B and B are deprecated in favor of B. The reason is that while B can be used as a drop-in replacement to these scripts, there is quite a lot of code duplication here. Individual scripts last around 150 lines of codes while the B program lasts 1200 lines, so they do a lot in addition of the common internals. The code duplication results in bugs occuring in both versions and needing two fixes. One example of such duplication are the bugs #1022216 in Debian and the issue #442 in GitHub that had the exact same fix, but one in B and the other B. In the long run, I would like to drop the individual scripts and only maintain one version of this code. The sure thing is that the individual scripts will not get improved anymore, so only B will get the new features. That being said, there is no deprecation urgency. I plan to keep the individual scripts as long as possible, and at least until 2030. If your project still use B and B in 2030, you may have a problem. We may also remove the deprecation of these scripts at some point, if a refactoring reduces the code duplication to zero. If you have an idea (or better: a patch), your help is welcome. =head2 What about the other translation tools for documentation using gettext? There are a few of them. Here is a possibly incomplete list, and more tools are coming at the horizon. =over =item B This is the tool developed by KDE people to handle DocBook XML. AFAIK, it was the first program to extract strings to translate from documentation to PO files, and inject them back after translation. It can only handle XML, and only a particular DTD. I'm quite unhappy with the handling of lists, which end in one big msgid. When the list become big, the chunk becomes harder to swallow. =item B This program done by Denis Barbier is a sort of precursor of the po4a SGML module, which more or less deprecates it. As the name says, it handles only the DebianDoc DTD, which is more or less a deprecated DTD. =item B Used by the GIMP Documentation Team since 2004, works quite well even if, as the name suggests, only with XML files and needs specially configured makefiles. =item B The Sphinx Documentation Project also uses gettext extensively to manage its translations. Unfortunately, it works only for a few text formats, rest and markdown, although it is perhaps the only tool that does this managing the whole translation process. =back The main advantages of po4a over them are the ease of extra content addition (which is even worse there) and the ability to achieve gettextization. =head2 SUMMARY of the advantages of the gettext based approach =over 2 =item The translations are not stored along with the original, which makes it possible to detect if translations become out of date. =item The translations are stored in separate files from each other, which prevents translators of different languages from interfering, both when submitting their patch and at the file encoding level. =item It is based internally on B (but B offers a very simple interface so that you don't need to understand the internals to use it). That way, we don't have to re-implement the wheel, and because of their wide use, we can think that these tools are more or less bug free. =item Nothing changed for the end-user (beside the fact translations will hopefully be better maintained). The resulting documentation file distributed is exactly the same. =item No need for translators to learn a new file syntax and their favorite PO file editor (like Emacs' PO mode, Lokalize or Gtranslator) will work just fine. =item gettext offers a simple way to get statistics about what is done, what should be reviewed and updated, and what is still to do. Some example can be found at those addresses: - https://docs.kde.org/stable5/en/kdesdk/lokalize/project-view.html - http://www.debian.org/intl/l10n/ =back But everything isn't green, and this approach also has some disadvantages we have to deal with. =over 2 =item Addenda are somewhat strange at the first glance. =item You can't adapt the translated text to your preferences, like splitting a paragraph here, and joining two other ones there. But in some sense, if there is an issue with the original, it should be reported as a bug anyway. =item Even with an easy interface, it remains a new tool people have to learn. One of my dreams would be to integrate somehow po4a to Gtranslator or Lokalize. When a documentation file is opened, the strings are automatically extracted, and a translated file + po file can be written to disk. If we manage to do an MS Word (TM) module (or at least RTF) professional translators may even use it. =back =head1 SEE ALSO =over =item The documentation of the all-in-one tool that you should use: L. =item The documentation of the individual po4a scripts: L, L, L, L. =item The additional helping scripts: L, L, L. =item The parsers of each formats, in particular to see the options accepted by each of them: L L, L, L, L, L, L, L, L, L, L, L, L, L, L, L, L, L, L, L. =item The implementation of the core infrastructure: L (particularly important to understand the code organization), L, L, L. Please also check the F file in the source tree. =back =head1 AUTHORS Denis Barbier Martin Quinson (mquinson#debian.org) =cut LocalWords: PO gettext SGML XML texinfo perl gettextize fr Lokalize KDE updatepo LocalWords: Gtranslator gettextization VCS regexp boundary LocalWords: lang TransTractor debconf diff poxml debiandoc LocalWords LocalWords: Denis barbier linuxfr org Quinson