summaryrefslogtreecommitdiffstats
path: root/src/jaegertracing/thrift/test/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'src/jaegertracing/thrift/test/README.md')
-rwxr-xr-xsrc/jaegertracing/thrift/test/README.md185
1 files changed, 185 insertions, 0 deletions
diff --git a/src/jaegertracing/thrift/test/README.md b/src/jaegertracing/thrift/test/README.md
new file mode 100755
index 000000000..19cc2a1a7
--- /dev/null
+++ b/src/jaegertracing/thrift/test/README.md
@@ -0,0 +1,185 @@
+# Apache Thrift - integration test suite
+
+This is the cross everything integration test suite for Apache Thrift.
+
+## Run
+
+### A. Using Make
+
+The test can be executed by:
+
+ make cross
+
+This starts the [test.py](test.py) script which does the real cross test with
+different transports, protocols and languages.
+
+Note that this skips any language that is not built locally. It also skips
+tests that are known to be failing. If you need more control over which tests
+to run, read following section.
+
+### B. Using test script directly
+
+Alternatively, you can invoke [test.py](test.py) directly. You need to run`make
+precross` once before executing it for the first time.
+
+For example, if you changed something in `nodejs` library and need to verify
+the patch, you can skip everything except `nodejs` itself and some reference
+implementation (currently `cpp` and `java` are recommended) like this:
+
+ ./configure --without-c_glib -without-csharp --without-erlang --without-lua ...
+ make precross -j8
+ test/test.py --server cpp,java --client nodejs
+ test/test.py --server nodejs --client cpp,java
+
+Another useful flag is --regex. For example, to run all tests that involve
+Java TBinaryProtocol:
+
+ test/test.py --regex "java.*binary"
+
+## Test case definition file
+
+The cross test cases are defined in [tests.json](tests.json).
+The root element is collection of test target definitions.
+Each test target definition looks like this:
+
+ {
+ "name": "somelib",
+
+ "client": {
+ "command": ["somelib_client_executable"],
+ "workdir": "somelib/bin",
+ "protocols": ["binary"],
+ "transports": ["buffered"],
+ "sockets": ["ip"],
+ },
+ "server": {
+ "command": ["somelib_server_executable"],
+ "workdir": "somelib/bin",
+ "protocols": ["binary"],
+ "transports": ["buffered"],
+ "sockets": ["ip", "ip-ssl"],
+ }
+ }
+
+Either client or server definition or both should be present.
+
+Parameters that are common to both `client` and `server` can be put to target
+definition root:
+
+ {
+ "name": "somelib",
+
+ "workdir": "somelib/bin",
+ "protocols": ["binary"],
+ "transports": ["buffered"],
+ "sockets": ["ip"],
+
+ "client": { "command": ["somelib_client_executable"] },
+ "server": {
+ "command": ["somelib_server_executable"],
+ "sockets": ["ip-ssl"]
+ }
+ }
+
+For the complete list of supported keys and their effect, see source code
+comment at the opt of [crossrunner/collect.py](crossrunner/collect.py).
+
+
+## List of known failures
+
+Since many cross tests currently fail (mainly due to partial incompatibility
+around exception handling), the test script specifically report for "not known
+before" failures.
+
+For this purpose, test cases known to (occasionally) fail are listed in
+`known_failures_<platform>.json` where `<platform>` matches with python
+`platform.system()` string.
+
+Currently, only Linux version is included.
+
+FYI, the file is initially generated by
+
+ test/test.py --update-expected-failures=overwrite
+
+after a full test run, then repeatedly
+
+ test/test.py --skip-known-failures
+ test/test.py --update-expected-failures=merge
+
+to update the known failures, run
+
+ make fail
+
+## Test executable specification
+
+### Command line parameters
+
+Unit tests for languages are usually located under lib/<lang>/test/
+cross language tests according to [ThriftTest.thrift](ThriftTest.thrift) shall be
+provided for every language including executables with the following command
+line interface:
+
+**Server command line interface:**
+
+ $ ./cpp/TestServer -h
+ Allowed options:
+ -h [ --help ] produce help message
+ --port arg (=9090) Port number to listen
+ --domain-socket arg Unix Domain Socket (e.g. /tmp/ThriftTest.thrift)
+ --named-pipe arg Windows Named Pipe (e.g. MyThriftPipe)
+ --server-type arg (=simple) type of server, "simple", "thread-pool",
+ "threaded", or "nonblocking"
+ --transport arg (=buffered) transport: buffered, framed, http, anonpipe
+ --protocol arg (=binary) protocol: binary, compact, json
+ --ssl Encrypted Transport using SSL
+ --processor-events processor-events
+ -n [ --workers ] arg (=4) Number of thread pools workers. Only valid for
+ thread-pool server type
+
+**Client command line interface:**
+
+ $ ./cpp/TestClient -h
+ Allowed options:
+ -h [ --help ] produce help message
+ --host arg (=localhost) Host to connect
+ --port arg (=9090) Port number to connect
+ --domain-socket arg Domain Socket (e.g. /tmp/ThriftTest.thrift),
+ instead of host and port
+ --named-pipe arg Windows Named Pipe (e.g. MyThriftPipe)
+ --anon-pipes hRead hWrite Windows Anonymous Pipes pair (handles)
+ --transport arg (=buffered) Transport: buffered, framed, http, evhttp
+ --protocol arg (=binary) Protocol: binary, compact, json
+ --ssl Encrypted Transport using SSL
+ -n [ --testloops ] arg (=1) Number of Tests
+ -t [ --threads ] arg (=1) Number of Test threads
+
+If you have executed the **make check** or **make cross** then you will be able to browse
+[gen-html/ThriftTest.html](gen-html/ThriftTest.html) with the test documentation.
+
+### Return code
+
+The return code (exit code) shall be 0 on success, or an integer in the range 1 - 255 on errors.
+In order to signal failed tests, the return code shall be composed from these bits to indicate
+failing tests:
+
+ #define TEST_BASETYPES 1 // 0000 0001
+ #define TEST_STRUCTS 2 // 0000 0010
+ #define TEST_CONTAINERS 4 // 0000 0100
+ #define TEST_EXCEPTIONS 8 // 0000 1000
+ #define TEST_UNKNOWN 64 // 0100 0000 (Failed to prepare environment etc.)
+ #define TEST_TIMEOUT 128 // 1000 0000
+ #define TEST_NOTUSED 48 // 0011 0000 (reserved bits)
+
+Tests that have not been executed at all count as errors.
+
+**Example:**
+
+During tests, the test client notices that some of the Struct tests fail.
+Furthermore, due to some other problem none of the Exception tests is executed.
+Therefore, the test client returns the code `10 = 2 | 8`, indicating the failure
+of both test 2 (TEST_STRUCTS) and test 8 (TEST_EXCEPTIONS).
+
+
+## SSL
+Test Keys and Certificates are provided in multiple formats under the following
+directory [test/keys](keys)