From c8bae7493d2f2910b57f13ded012e86bdcfb0532 Mon Sep 17 00:00:00 2001 From: Daniel Baumann Date: Sun, 7 Apr 2024 16:47:53 +0200 Subject: Adding upstream version 1:2.39.2. Signed-off-by: Daniel Baumann --- Documentation/technical/.gitignore | 1 + Documentation/technical/api-error-handling.txt | 103 ++ Documentation/technical/api-index-skel.txt | 13 + Documentation/technical/api-index.sh | 28 + Documentation/technical/api-merge.txt | 36 + Documentation/technical/api-parse-options.txt | 349 +++++ Documentation/technical/api-simple-ipc.txt | 105 ++ Documentation/technical/api-trace2.txt | 1339 ++++++++++++++++++++ Documentation/technical/bitmap-format.txt | 257 ++++ Documentation/technical/bundle-uri.txt | 572 +++++++++ Documentation/technical/commit-graph.txt | 401 ++++++ .../technical/directory-rename-detection.txt | 118 ++ .../technical/hash-function-transition.txt | 830 ++++++++++++ .../technical/long-running-process-protocol.txt | 50 + Documentation/technical/multi-pack-index.txt | 100 ++ Documentation/technical/pack-heuristics.txt | 460 +++++++ Documentation/technical/packfile-uri.txt | 82 ++ Documentation/technical/parallel-checkout.txt | 270 ++++ Documentation/technical/partial-clone.txt | 367 ++++++ Documentation/technical/racy-git.txt | 201 +++ Documentation/technical/reftable.txt | 1098 ++++++++++++++++ Documentation/technical/remembering-renames.txt | 671 ++++++++++ Documentation/technical/repository-version.txt | 102 ++ Documentation/technical/rerere.txt | 186 +++ Documentation/technical/scalar.txt | 66 + Documentation/technical/send-pack-pipeline.txt | 63 + Documentation/technical/shallow.txt | 60 + Documentation/technical/sparse-checkout.txt | 1103 ++++++++++++++++ Documentation/technical/sparse-index.txt | 208 +++ Documentation/technical/trivial-merge.txt | 121 ++ 30 files changed, 9360 insertions(+) create mode 100644 Documentation/technical/.gitignore create mode 100644 Documentation/technical/api-error-handling.txt create mode 100644 Documentation/technical/api-index-skel.txt create mode 100755 Documentation/technical/api-index.sh create mode 100644 Documentation/technical/api-merge.txt create mode 100644 Documentation/technical/api-parse-options.txt create mode 100644 Documentation/technical/api-simple-ipc.txt create mode 100644 Documentation/technical/api-trace2.txt create mode 100644 Documentation/technical/bitmap-format.txt create mode 100644 Documentation/technical/bundle-uri.txt create mode 100644 Documentation/technical/commit-graph.txt create mode 100644 Documentation/technical/directory-rename-detection.txt create mode 100644 Documentation/technical/hash-function-transition.txt create mode 100644 Documentation/technical/long-running-process-protocol.txt create mode 100644 Documentation/technical/multi-pack-index.txt create mode 100644 Documentation/technical/pack-heuristics.txt create mode 100644 Documentation/technical/packfile-uri.txt create mode 100644 Documentation/technical/parallel-checkout.txt create mode 100644 Documentation/technical/partial-clone.txt create mode 100644 Documentation/technical/racy-git.txt create mode 100644 Documentation/technical/reftable.txt create mode 100644 Documentation/technical/remembering-renames.txt create mode 100644 Documentation/technical/repository-version.txt create mode 100644 Documentation/technical/rerere.txt create mode 100644 Documentation/technical/scalar.txt create mode 100644 Documentation/technical/send-pack-pipeline.txt create mode 100644 Documentation/technical/shallow.txt create mode 100644 Documentation/technical/sparse-checkout.txt create mode 100644 Documentation/technical/sparse-index.txt create mode 100644 Documentation/technical/trivial-merge.txt (limited to 'Documentation/technical') diff --git a/Documentation/technical/.gitignore b/Documentation/technical/.gitignore new file mode 100644 index 0000000..8aa891d --- /dev/null +++ b/Documentation/technical/.gitignore @@ -0,0 +1 @@ +api-index.txt diff --git a/Documentation/technical/api-error-handling.txt b/Documentation/technical/api-error-handling.txt new file mode 100644 index 0000000..665c496 --- /dev/null +++ b/Documentation/technical/api-error-handling.txt @@ -0,0 +1,103 @@ +Error reporting in git +====================== + +`BUG`, `bug`, `die`, `usage`, `error`, and `warning` report errors of +various kinds. + +- `BUG` is for failed internal assertions that should never happen, + i.e. a bug in git itself. + +- `bug` (lower-case, not `BUG`) is supposed to be used like `BUG` but + prints a "BUG" message instead of calling `abort()`. ++ +A call to `bug()` will then result in a "real" call to the `BUG()` +function, either explicitly by invoking `BUG_if_bug()` after call(s) +to `bug()`, or implicitly at `exit()` time where we'll check if we +encountered any outstanding `bug()` invocations. ++ +If there were no prior calls to `bug()` before invoking `BUG_if_bug()` +the latter is a NOOP. The `BUG_if_bug()` function takes the same +arguments as `BUG()` itself. Calling `BUG_if_bug()` explicitly isn't +necessary, but ensures that we die as soon as possible. ++ +If you know you had prior calls to `bug()` then calling `BUG()` itself +is equivalent to calling `BUG_if_bug()`, the latter being a wrapper +calling `BUG()` if we've set a flag indicating that we've called +`bug()`. ++ +This is for the convenience of APIs who'd like to potentially report +more than one "bug", such as the optbug() validation in +parse-options.c. + +- `die` is for fatal application errors. It prints a message to + the user and exits with status 128. + +- `usage` is for errors in command line usage. After printing its + message, it exits with status 129. (See also `usage_with_options` + in the link:api-parse-options.html[parse-options API].) + +- `error` is for non-fatal library errors. It prints a message + to the user and returns -1 for convenience in signaling the error + to the caller. + +- `warning` is for reporting situations that probably should not + occur but which the user (and Git) can continue to work around + without running into too many problems. Like `error`, it + returns -1 after reporting the situation to the caller. + +These reports will be logged via the trace2 facility. See the "error" +event in link:api-trace2.html[trace2 API]. + +Customizable error handlers +--------------------------- + +The default behavior of `die` and `error` is to write a message to +stderr and then exit or return as appropriate. This behavior can be +overridden using `set_die_routine` and `set_error_routine`. For +example, "git daemon" uses set_die_routine to write the reason `die` +was called to syslog before exiting. + +Library errors +-------------- + +Functions return a negative integer on error. Details beyond that +vary from function to function: + +- Some functions return -1 for all errors. Others return a more + specific value depending on how the caller might want to react + to the error. + +- Some functions report the error to stderr with `error`, + while others leave that for the caller to do. + +- errno is not meaningful on return from most functions (except + for thin wrappers for system calls). + +Check the function's API documentation to be sure. + +Caller-handled errors +--------------------- + +An increasing number of functions take a parameter 'struct strbuf *err'. +On error, such functions append a message about what went wrong to the +'err' strbuf. The message is meant to be complete enough to be passed +to `die` or `error` as-is. For example: + + if (ref_transaction_commit(transaction, &err)) + die("%s", err.buf); + +The 'err' parameter will be untouched if no error occurred, so multiple +function calls can be chained: + + t = ref_transaction_begin(&err); + if (!t || + ref_transaction_update(t, "HEAD", ..., &err) || + ret_transaction_commit(t, &err)) + die("%s", err.buf); + +The 'err' parameter must be a pointer to a valid strbuf. To silence +a message, pass a strbuf that is explicitly ignored: + + if (thing_that_can_fail_in_an_ignorable_way(..., &err)) + /* This failure is okay. */ + strbuf_reset(&err); diff --git a/Documentation/technical/api-index-skel.txt b/Documentation/technical/api-index-skel.txt new file mode 100644 index 0000000..eda8c19 --- /dev/null +++ b/Documentation/technical/api-index-skel.txt @@ -0,0 +1,13 @@ +Git API Documents +================= + +Git has grown a set of internal API over time. This collection +documents them. + +//////////////////////////////////////////////////////////////// +// table of contents begin +//////////////////////////////////////////////////////////////// + +//////////////////////////////////////////////////////////////// +// table of contents end +//////////////////////////////////////////////////////////////// diff --git a/Documentation/technical/api-index.sh b/Documentation/technical/api-index.sh new file mode 100755 index 0000000..9c3f413 --- /dev/null +++ b/Documentation/technical/api-index.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +( + c=//////////////////////////////////////////////////////////////// + skel=api-index-skel.txt + sed -e '/^\/\/ table of contents begin/q' "$skel" + echo "$c" + + ls api-*.txt | + while read filename + do + case "$filename" in + api-index-skel.txt | api-index.txt) continue ;; + esac + title=$(sed -e 1q "$filename") + html=${filename%.txt}.html + echo "* link:$html[$title]" + done + echo "$c" + sed -n -e '/^\/\/ table of contents end/,$p' "$skel" +) >api-index.txt+ + +if test -f api-index.txt && cmp api-index.txt api-index.txt+ >/dev/null +then + rm -f api-index.txt+ +else + mv api-index.txt+ api-index.txt +fi diff --git a/Documentation/technical/api-merge.txt b/Documentation/technical/api-merge.txt new file mode 100644 index 0000000..487d4d8 --- /dev/null +++ b/Documentation/technical/api-merge.txt @@ -0,0 +1,36 @@ +merge API +========= + +The merge API helps a program to reconcile two competing sets of +improvements to some files (e.g., unregistered changes from the work +tree versus changes involved in switching to a new branch), reporting +conflicts if found. The library called through this API is +responsible for a few things. + + * determining which trees to merge (recursive ancestor consolidation); + + * lining up corresponding files in the trees to be merged (rename + detection, subtree shifting), reporting edge cases like add/add + and rename/rename conflicts to the user; + + * performing a three-way merge of corresponding files, taking + path-specific merge drivers (specified in `.gitattributes`) + into account. + +Data structures +--------------- + +* `mmbuffer_t`, `mmfile_t` + +These store data usable for use by the xdiff backend, for writing and +for reading, respectively. See `xdiff/xdiff.h` for the definitions +and `diff.c` for examples. + +* `struct ll_merge_options` + +Check ll-merge.h for details. + +Low-level (single file) merge +----------------------------- + +Check ll-merge.h for details. diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt new file mode 100644 index 0000000..61fa6ee --- /dev/null +++ b/Documentation/technical/api-parse-options.txt @@ -0,0 +1,349 @@ +parse-options API +================= + +The parse-options API is used to parse and massage options in Git +and to provide a usage help with consistent look. + +Basics +------ + +The argument vector `argv[]` may usually contain mandatory or optional +'non-option arguments', e.g. a filename or a branch, 'options', and +'subcommands'. +Options are optional arguments that start with a dash and +that allow to change the behavior of a command. + +* There are basically three types of options: + 'boolean' options, + options with (mandatory) 'arguments' and + options with 'optional arguments' + (i.e. a boolean option that can be adjusted). + +* There are basically two forms of options: + 'Short options' consist of one dash (`-`) and one alphanumeric + character. + 'Long options' begin with two dashes (`--`) and some + alphanumeric characters. + +* Options are case-sensitive. + Please define 'lower-case long options' only. + +The parse-options API allows: + +* 'stuck' and 'separate form' of options with arguments. + `-oArg` is stuck, `-o Arg` is separate form. + `--option=Arg` is stuck, `--option Arg` is separate form. + +* Long options may be 'abbreviated', as long as the abbreviation + is unambiguous. + +* Short options may be bundled, e.g. `-a -b` can be specified as `-ab`. + +* Boolean long options can be 'negated' (or 'unset') by prepending + `no-`, e.g. `--no-abbrev` instead of `--abbrev`. Conversely, + options that begin with `no-` can be 'negated' by removing it. + Other long options can be unset (e.g., set string to NULL, set + integer to 0) by prepending `no-`. + +* Options and non-option arguments can clearly be separated using the `--` + option, e.g. `-a -b --option -- --this-is-a-file` indicates that + `--this-is-a-file` must not be processed as an option. + +Subcommands are special in a couple of ways: + +* Subcommands only have long form, and they have no double dash prefix, no + negated form, and no description, and they don't take any arguments, and + can't be abbreviated. + +* There must be exactly one subcommand among the arguments, or zero if the + command has a default operation mode. + +* All arguments following the subcommand are considered to be arguments of + the subcommand, and, conversely, arguments meant for the subcommand may + not precede the subcommand. + +Therefore, if the options array contains at least one subcommand and +`parse_options()` encounters the first dashless argument, it will either: + +* stop and return, if that dashless argument is a known subcommand, setting + `value` to the function pointer associated with that subcommand, storing + the name of the subcommand in argv[0], and leaving the rest of the + arguments unprocessed, or + +* stop and return, if it was invoked with the `PARSE_OPT_SUBCOMMAND_OPTIONAL` + flag and that dashless argument doesn't match any subcommands, leaving + `value` unchanged and the rest of the arguments unprocessed, or + +* show error and usage, and abort. + +Steps to parse options +---------------------- + +. `#include "parse-options.h"` + +. define a NULL-terminated + `static const char * const builtin_foo_usage[]` array + containing alternative usage strings + +. define `builtin_foo_options` array as described below + in section 'Data Structure'. + +. in `cmd_foo(int argc, const char **argv, const char *prefix)` + call + + argc = parse_options(argc, argv, prefix, builtin_foo_options, builtin_foo_usage, flags); ++ +`parse_options()` will filter out the processed options of `argv[]` and leave the +non-option arguments in `argv[]`. +`argc` is updated appropriately because of the assignment. ++ +You can also pass NULL instead of a usage array as the fifth parameter of +parse_options(), to avoid displaying a help screen with usage info and +option list. This should only be done if necessary, e.g. to implement +a limited parser for only a subset of the options that needs to be run +before the full parser, which in turn shows the full help message. ++ +Flags are the bitwise-or of: + +`PARSE_OPT_KEEP_DASHDASH`:: + Keep the `--` that usually separates options from + non-option arguments. + +`PARSE_OPT_STOP_AT_NON_OPTION`:: + Usually the whole argument vector is massaged and reordered. + Using this flag, processing is stopped at the first non-option + argument. + +`PARSE_OPT_KEEP_ARGV0`:: + Keep the first argument, which contains the program name. It's + removed from argv[] by default. + +`PARSE_OPT_KEEP_UNKNOWN_OPT`:: + Keep unknown options instead of erroring out. This doesn't + work for all combinations of arguments as users might expect + it to do. E.g. if the first argument in `--unknown --known` + takes a value (which we can't know), the second one is + mistakenly interpreted as a known option. Similarly, if + `PARSE_OPT_STOP_AT_NON_OPTION` is set, the second argument in + `--unknown value` will be mistakenly interpreted as a + non-option, not as a value belonging to the unknown option, + the parser early. That's why parse_options() errors out if + both options are set. + Note that non-option arguments are always kept, even without + this flag. + +`PARSE_OPT_NO_INTERNAL_HELP`:: + By default, parse_options() handles `-h`, `--help` and + `--help-all` internally, by showing a help screen. This option + turns it off and allows one to add custom handlers for these + options, or to just leave them unknown. + +`PARSE_OPT_SUBCOMMAND_OPTIONAL`:: + Don't error out when no subcommand is specified. + +Note that `PARSE_OPT_STOP_AT_NON_OPTION` is incompatible with subcommands; +while `PARSE_OPT_KEEP_DASHDASH` and `PARSE_OPT_KEEP_UNKNOWN_OPT` can only be +used with subcommands when combined with `PARSE_OPT_SUBCOMMAND_OPTIONAL`. + +Data Structure +-------------- + +The main data structure is an array of the `option` struct, +say `static struct option builtin_add_options[]`. +There are some macros to easily define options: + +`OPT__ABBREV(&int_var)`:: + Add `--abbrev[=]`. + +`OPT__COLOR(&int_var, description)`:: + Add `--color[=]` and `--no-color`. + +`OPT__DRY_RUN(&int_var, description)`:: + Add `-n, --dry-run`. + +`OPT__FORCE(&int_var, description)`:: + Add `-f, --force`. + +`OPT__QUIET(&int_var, description)`:: + Add `-q, --quiet`. + +`OPT__VERBOSE(&int_var, description)`:: + Add `-v, --verbose`. + +`OPT_GROUP(description)`:: + Start an option group. `description` is a short string that + describes the group or an empty string. + Start the description with an upper-case letter. + +`OPT_BOOL(short, long, &int_var, description)`:: + Introduce a boolean option. `int_var` is set to one with + `--option` and set to zero with `--no-option`. + +`OPT_COUNTUP(short, long, &int_var, description)`:: + Introduce a count-up option. + Each use of `--option` increments `int_var`, starting from zero + (even if initially negative), and `--no-option` resets it to + zero. To determine if `--option` or `--no-option` was encountered at + all, initialize `int_var` to a negative value, and if it is still + negative after parse_options(), then neither `--option` nor + `--no-option` was seen. + +`OPT_BIT(short, long, &int_var, description, mask)`:: + Introduce a boolean option. + If used, `int_var` is bitwise-ored with `mask`. + +`OPT_NEGBIT(short, long, &int_var, description, mask)`:: + Introduce a boolean option. + If used, `int_var` is bitwise-anded with the inverted `mask`. + +`OPT_SET_INT(short, long, &int_var, description, integer)`:: + Introduce an integer option. + `int_var` is set to `integer` with `--option`, and + reset to zero with `--no-option`. + +`OPT_STRING(short, long, &str_var, arg_str, description)`:: + Introduce an option with string argument. + The string argument is put into `str_var`. + +`OPT_STRING_LIST(short, long, &struct string_list, arg_str, description)`:: + Introduce an option with string argument. + The string argument is stored as an element in `string_list`. + Use of `--no-option` will clear the list of preceding values. + +`OPT_INTEGER(short, long, &int_var, description)`:: + Introduce an option with integer argument. + The integer is put into `int_var`. + +`OPT_MAGNITUDE(short, long, &unsigned_long_var, description)`:: + Introduce an option with a size argument. The argument must be a + non-negative integer and may include a suffix of 'k', 'm' or 'g' to + scale the provided value by 1024, 1024^2 or 1024^3 respectively. + The scaled value is put into `unsigned_long_var`. + +`OPT_EXPIRY_DATE(short, long, ×tamp_t_var, description)`:: + Introduce an option with expiry date argument, see `parse_expiry_date()`. + The timestamp is put into `timestamp_t_var`. + +`OPT_CALLBACK(short, long, &var, arg_str, description, func_ptr)`:: + Introduce an option with argument. + The argument will be fed into the function given by `func_ptr` + and the result will be put into `var`. + See 'Option Callbacks' below for a more elaborate description. + +`OPT_FILENAME(short, long, &var, description)`:: + Introduce an option with a filename argument. + The filename will be prefixed by passing the filename along with + the prefix argument of `parse_options()` to `prefix_filename()`. + +`OPT_NUMBER_CALLBACK(&var, description, func_ptr)`:: + Recognize numerical options like -123 and feed the integer as + if it was an argument to the function given by `func_ptr`. + The result will be put into `var`. There can be only one such + option definition. It cannot be negated and it takes no + arguments. Short options that happen to be digits take + precedence over it. + +`OPT_COLOR_FLAG(short, long, &int_var, description)`:: + Introduce an option that takes an optional argument that can + have one of three values: "always", "never", or "auto". If the + argument is not given, it defaults to "always". The `--no-` form + works like `--long=never`; it cannot take an argument. If + "always", set `int_var` to 1; if "never", set `int_var` to 0; if + "auto", set `int_var` to 1 if stdout is a tty or a pager, + 0 otherwise. + +`OPT_NOOP_NOARG(short, long)`:: + Introduce an option that has no effect and takes no arguments. + Use it to hide deprecated options that are still to be recognized + and ignored silently. + +`OPT_PASSTHRU(short, long, &char_var, arg_str, description, flags)`:: + Introduce an option that will be reconstructed into a char* string, + which must be initialized to NULL. This is useful when you need to + pass the command-line option to another command. Any previous value + will be overwritten, so this should only be used for options where + the last one specified on the command line wins. + +`OPT_PASSTHRU_ARGV(short, long, &strvec_var, arg_str, description, flags)`:: + Introduce an option where all instances of it on the command-line will + be reconstructed into a strvec. This is useful when you need to + pass the command-line option, which can be specified multiple times, + to another command. + +`OPT_CMDMODE(short, long, &int_var, description, enum_val)`:: + Define an "operation mode" option, only one of which in the same + group of "operating mode" options that share the same `int_var` + can be given by the user. `int_var` is set to `enum_val` when the + option is used, but an error is reported if other "operating mode" + option has already set its value to the same `int_var`. + In new commands consider using subcommands instead. + +`OPT_SUBCOMMAND(long, &fn_ptr, subcommand_fn)`:: + Define a subcommand. `subcommand_fn` is put into `fn_ptr` when + this subcommand is used. + +The last element of the array must be `OPT_END()`. + +If not stated otherwise, interpret the arguments as follows: + +* `short` is a character for the short option + (e.g. `'e'` for `-e`, use `0` to omit), + +* `long` is a string for the long option + (e.g. `"example"` for `--example`, use `NULL` to omit), + +* `int_var` is an integer variable, + +* `str_var` is a string variable (`char *`), + +* `arg_str` is the string that is shown as argument + (e.g. `"branch"` will result in ``). + If set to `NULL`, three dots (`...`) will be displayed. + +* `description` is a short string to describe the effect of the option. + It shall begin with a lower-case letter and a full stop (`.`) shall be + omitted at the end. + +Option Callbacks +---------------- + +The function must be defined in this form: + + int func(const struct option *opt, const char *arg, int unset) + +The callback mechanism is as follows: + +* Inside `func`, the only interesting member of the structure + given by `opt` is the void pointer `opt->value`. + `*opt->value` will be the value that is saved into `var`, if you + use `OPT_CALLBACK()`. + For example, do `*(unsigned long *)opt->value = 42;` to get 42 + into an `unsigned long` variable. + +* Return value `0` indicates success and non-zero return + value will invoke `usage_with_options()` and, thus, die. + +* If the user negates the option, `arg` is `NULL` and `unset` is 1. + +Sophisticated option parsing +---------------------------- + +If you need, for example, option callbacks with optional arguments +or without arguments at all, or if you need other special cases, +that are not handled by the macros above, you need to specify the +members of the `option` structure manually. + +This is not covered in this document, but well documented +in `parse-options.h` itself. + +Examples +-------- + +See `test-parse-options.c` and +`builtin/add.c`, +`builtin/clone.c`, +`builtin/commit.c`, +`builtin/fetch.c`, +`builtin/fsck.c`, +`builtin/rm.c` +for real-world examples. diff --git a/Documentation/technical/api-simple-ipc.txt b/Documentation/technical/api-simple-ipc.txt new file mode 100644 index 0000000..d44ada9 --- /dev/null +++ b/Documentation/technical/api-simple-ipc.txt @@ -0,0 +1,105 @@ +Simple-IPC API +============== + +The Simple-IPC API is a collection of `ipc_` prefixed library routines +and a basic communication protocol that allow an IPC-client process to +send an application-specific IPC-request message to an IPC-server +process and receive an application-specific IPC-response message. + +Communication occurs over a named pipe on Windows and a Unix domain +socket on other platforms. IPC-clients and IPC-servers rendezvous at +a previously agreed-to application-specific pathname (which is outside +the scope of this design) that is local to the computer system. + +The IPC-server routines within the server application process create a +thread pool to listen for connections and receive request messages +from multiple concurrent IPC-clients. When received, these messages +are dispatched up to the server application callbacks for handling. +IPC-server routines then incrementally relay responses back to the +IPC-client. + +The IPC-client routines within a client application process connect +to the IPC-server and send a request message and wait for a response. +When received, the response is returned back the caller. + +For example, the `fsmonitor--daemon` feature will be built as a server +application on top of the IPC-server library routines. It will have +threads watching for file system events and a thread pool waiting for +client connections. Clients, such as `git status` will request a list +of file system events since a point in time and the server will +respond with a list of changed files and directories. The formats of +the request and response are application-specific; the IPC-client and +IPC-server routines treat them as opaque byte streams. + + +Comparison with sub-process model +--------------------------------- + +The Simple-IPC mechanism differs from the existing `sub-process.c` +model (Documentation/technical/long-running-process-protocol.txt) and +used by applications like Git-LFS. In the LFS-style sub-process model +the helper is started by the foreground process, communication happens +via a pair of file descriptors bound to the stdin/stdout of the +sub-process, the sub-process only serves the current foreground +process, and the sub-process exits when the foreground process +terminates. + +In the Simple-IPC model the server is a very long-running service. It +can service many clients at the same time and has a private socket or +named pipe connection to each active client. It might be started +(on-demand) by the current client process or it might have been +started by a previous client or by the OS at boot time. The server +process is not associated with a terminal and it persists after +clients terminate. Clients do not have access to the stdin/stdout of +the server process and therefore must communicate over sockets or +named pipes. + + +Server startup and shutdown +--------------------------- + +How an application server based upon IPC-server is started is also +outside the scope of the Simple-IPC design and is a property of the +application using it. For example, the server might be started or +restarted during routine maintenance operations, or it might be +started as a system service during the system boot-up sequence, or it +might be started on-demand by a foreground Git command when needed. + +Similarly, server shutdown is a property of the application using +the simple-ipc routines. For example, the server might decide to +shutdown when idle or only upon explicit request. + + +Simple-IPC protocol +------------------- + +The Simple-IPC protocol consists of a single request message from the +client and an optional response message from the server. Both the +client and server messages are unlimited in length and are terminated +with a flush packet. + +The pkt-line routines (linkgit:gitprotocol-common[5]) +are used to simplify buffer management during message generation, +transmission, and reception. A flush packet is used to mark the end +of the message. This allows the sender to incrementally generate and +transmit the message. It allows the receiver to incrementally receive +the message in chunks and to know when they have received the entire +message. + +The actual byte format of the client request and server response +messages are application specific. The IPC layer transmits and +receives them as opaque byte buffers without any concern for the +content within. It is the job of the calling application layer to +understand the contents of the request and response messages. + + +Summary +------- + +Conceptually, the Simple-IPC protocol is similar to an HTTP REST +request. Clients connect, make an application-specific and +stateless request, receive an application-specific +response, and disconnect. It is a one round trip facility for +querying the server. The Simple-IPC routines hide the socket, +named pipe, and thread pool details and allow the application +layer to focus on the application at hand. diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt new file mode 100644 index 0000000..de5fc25 --- /dev/null +++ b/Documentation/technical/api-trace2.txt @@ -0,0 +1,1339 @@ += Trace2 API + +The Trace2 API can be used to print debug, performance, and telemetry +information to stderr or a file. The Trace2 feature is inactive unless +explicitly enabled by enabling one or more Trace2 Targets. + +The Trace2 API is intended to replace the existing (Trace1) +`printf()`-style tracing provided by the existing `GIT_TRACE` and +`GIT_TRACE_PERFORMANCE` facilities. During initial implementation, +Trace2 and Trace1 may operate in parallel. + +The Trace2 API defines a set of high-level messages with known fields, +such as (`start`: `argv`) and (`exit`: {`exit-code`, `elapsed-time`}). + +Trace2 instrumentation throughout the Git code base sends Trace2 +messages to the enabled Trace2 Targets. Targets transform these +messages content into purpose-specific formats and write events to +their data streams. In this manner, the Trace2 API can drive +many different types of analysis. + +Targets are defined using a VTable allowing easy extension to other +formats in the future. This might be used to define a binary format, +for example. + +Trace2 is controlled using `trace2.*` config values in the system and +global config files and `GIT_TRACE2*` environment variables. Trace2 does +not read from repo local or worktree config files, nor does it respect +`-c` command line config settings. + +== Trace2 Targets + +Trace2 defines the following set of Trace2 Targets. +Format details are given in a later section. + +=== The Normal Format Target + +The normal format target is a traditional `printf()` format and similar +to the `GIT_TRACE` format. This format is enabled with the `GIT_TRACE2` +environment variable or the `trace2.normalTarget` system or global +config setting. + +For example + +------------ +$ export GIT_TRACE2=~/log.normal +$ git version +git version 2.20.1.155.g426c96fcdb +------------ + +or + +------------ +$ git config --global trace2.normalTarget ~/log.normal +$ git version +git version 2.20.1.155.g426c96fcdb +------------ + +yields + +------------ +$ cat ~/log.normal +12:28:42.620009 common-main.c:38 version 2.20.1.155.g426c96fcdb +12:28:42.620989 common-main.c:39 start git version +12:28:42.621101 git.c:432 cmd_name version (version) +12:28:42.621215 git.c:662 exit elapsed:0.001227 code:0 +12:28:42.621250 trace2/tr2_tgt_normal.c:124 atexit elapsed:0.001265 code:0 +------------ + +=== The Performance Format Target + +The performance format target (PERF) is a column-based format to +replace `GIT_TRACE_PERFORMANCE` and is suitable for development and +testing, possibly to complement tools like `gprof`. This format is +enabled with the `GIT_TRACE2_PERF` environment variable or the +`trace2.perfTarget` system or global config setting. + +For example + +------------ +$ export GIT_TRACE2_PERF=~/log.perf +$ git version +git version 2.20.1.155.g426c96fcdb +------------ + +or + +------------ +$ git config --global trace2.perfTarget ~/log.perf +$ git version +git version 2.20.1.155.g426c96fcdb +------------ + +yields + +------------ +$ cat ~/log.perf +12:28:42.620675 common-main.c:38 | d0 | main | version | | | | | 2.20.1.155.g426c96fcdb +12:28:42.621001 common-main.c:39 | d0 | main | start | | 0.001173 | | | git version +12:28:42.621111 git.c:432 | d0 | main | cmd_name | | | | | version (version) +12:28:42.621225 git.c:662 | d0 | main | exit | | 0.001227 | | | code:0 +12:28:42.621259 trace2/tr2_tgt_perf.c:211 | d0 | main | atexit | | 0.001265 | | | code:0 +------------ + +=== The Event Format Target + +The event format target is a JSON-based format of event data suitable +for telemetry analysis. This format is enabled with the `GIT_TRACE2_EVENT` +environment variable or the `trace2.eventTarget` system or global config +setting. + +For example + +------------ +$ export GIT_TRACE2_EVENT=~/log.event +$ git version +git version 2.20.1.155.g426c96fcdb +------------ + +or + +------------ +$ git config --global trace2.eventTarget ~/log.event +$ git version +git version 2.20.1.155.g426c96fcdb +------------ + +yields + +------------ +$ cat ~/log.event +{"event":"version","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.620713Z","file":"common-main.c","line":38,"evt":"3","exe":"2.20.1.155.g426c96fcdb"} +{"event":"start","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621027Z","file":"common-main.c","line":39,"t_abs":0.001173,"argv":["git","version"]} +{"event":"cmd_name","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621122Z","file":"git.c","line":432,"name":"version","hierarchy":"version"} +{"event":"exit","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621236Z","file":"git.c","line":662,"t_abs":0.001227,"code":0} +{"event":"atexit","sid":"20190408T191610.507018Z-H9b68c35f-P000059a8","thread":"main","time":"2019-01-16T17:28:42.621268Z","file":"trace2/tr2_tgt_event.c","line":163,"t_abs":0.001265,"code":0} +------------ + +=== Enabling a Target + +To enable a target, set the corresponding environment variable or +system or global config value to one of the following: + +include::../trace2-target-values.txt[] + +When trace files are written to a target directory, they will be named according +to the last component of the SID (optionally followed by a counter to avoid +filename collisions). + +== Trace2 API + +The Trace2 public API is defined and documented in `trace2.h`; refer to it for +more information. All public functions and macros are prefixed +with `trace2_` and are implemented in `trace2.c`. + +There are no public Trace2 data structures. + +The Trace2 code also defines a set of private functions and data types +in the `trace2/` directory. These symbols are prefixed with `tr2_` +and should only be used by functions in `trace2.c` (or other private +source files in `trace2/`). + +=== Conventions for Public Functions and Macros + +Some functions have a `_fl()` suffix to indicate that they take `file` +and `line-number` arguments. + +Some functions have a `_va_fl()` suffix to indicate that they also +take a `va_list` argument. + +Some functions have a `_printf_fl()` suffix to indicate that they also +take a `printf()` style format with a variable number of arguments. + +CPP wrapper macros are defined to hide most of these details. + +== Trace2 Target Formats + +=== NORMAL Format + +Events are written as lines of the form: + +------------ +[