.. Licensed to the Apache Software Foundation (ASF) under one .. or more contributor license agreements. See the NOTICE file .. distributed with this work for additional information .. regarding copyright ownership. The ASF licenses this file .. to you under the Apache License, Version 2.0 (the .. "License"); you may not use this file except in compliance .. with the License. You may obtain a copy of the License at .. http://www.apache.org/licenses/LICENSE-2.0 .. Unless required by applicable law or agreed to in writing, .. software distributed under the License is distributed on an .. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY .. KIND, either express or implied. See the License for the .. specific language governing permissions and limitations .. under the License. .. WARNING .. This is an auto-generated file. Please do not edit. .. To reproduce, please run :code:`./make_data_model_rst.sh`. .. (This requires you have the .. `psql client `_ .. and have started the docker containers using .. :code:`docker-compose up`). .. _benchmark-data-model: Benchmark data model ==================== .. graphviz:: data_model.dot .. _benchmark-ingestion: Benchmark ingestion helper functions ==================================== ingest_benchmark_run_view ------------------------- :code:`ingest_benchmark_run_view(from_jsonb jsonb)` The argument is a JSON object. NOTE: key names must be entirely lowercase, or the insert will fail. Extra key-value pairs are ignored. Example:: [ { "benchmark_name": "Benchmark 2", "benchmark_version": "version 0", "parameter_values": {"arg0": 100, "arg1": 5}, "value": 2.5, "git_commit_timestamp": "2019-02-08 22:35:53 +0100", "git_hash": "324d3cf198444a", "val_min": 1, "val_q1": 2, "val_q3": 3, "val_max": 4, "std_dev": 1.41, "n_obs": 8, "run_timestamp": "2019-02-14 03:00:05 -0600", "mac_address": "08:00:2b:01:02:03", "benchmark_language": "Python", "language_implementation_version": "CPython 2.7", "dependencies": {"six": "", "numpy": "1.14", "other_lib": "1.0"} }, { "benchmark_name": "Benchmark 2", "benchmark_version": "version 0", "parameter_values": {"arg0": 1000, "arg1": 5}, "value": 5, "git_commit_timestamp": "2019-02-08 22:35:53 +0100", "git_hash": "324d3cf198444a", "std_dev": 3.14, "n_obs": 8, "run_timestamp": "2019-02-14 03:00:10 -0600", "mac_address": "08:00:2b:01:02:03", "benchmark_language": "Python", "language_implementation_version": "CPython 2.7", "dependencies": {"six": "", "numpy": "1.14", "other_lib": "1.0"} } ] To identify which columns in "benchmark_run_view" are required, please see the view documentation in :ref:`benchmark-data-model`. back to `Benchmark data model `_ ingest_benchmark_view --------------------- :code:`ingest_benchmark_view(from_jsonb jsonb)` The argument is a JSON object. NOTE: key names must be entirely lowercase, or the insert will fail. Extra key-value pairs are ignored. Example:: [ { "benchmark_name": "Benchmark 1", "parameter_names": ["arg0", "arg1", "arg2"], "benchmark_description": "First benchmark", "benchmark_type": "Time", "units": "miliseconds", "lessisbetter": true, "benchmark_version": "second version", "benchmark_language": "Python" }, { "benchmark_name": "Benchmark 2", "parameter_names": ["arg0", "arg1"], "benchmark_description": "Description 2.", "benchmark_type": "Time", "units": "nanoseconds", "lessisbetter": true, "benchmark_version": "second version", "benchmark_language": "Python" } ] To identify which columns in "benchmark_view" are required, please see the view documentation in :ref:`benchmark-data-model`. back to `Benchmark data model `_ ingest_benchmark_runs_with_context ---------------------------------- :code:`ingest_benchmark_runs_with_context(from_jsonb jsonb)` The argument is a JSON object. NOTE: key names must be entirely lowercase, or the insert will fail. Extra key-value pairs are ignored. The object contains three key-value pairs:: {"context": { "mac_address": "08:00:2b:01:02:03", "benchmark_language": "Python", "language_implementation_version": "CPython 3.6", "dependencies": {"six": "", "numpy": "1.14", "other_lib": "1.0"}, "git_commit_timestamp": "2019-02-14 22:42:22 +0100", "git_hash": "123456789abcde", "run_timestamp": "2019-02-14 03:00:40 -0600", "extra stuff": "does not hurt anything and will not be added." }, "benchmark_version": { "Benchmark Name 1": "Any string can be a version.", "Benchmark Name 2": "A git hash can be a version.", "An Unused Benchmark Name": "Will be ignored." }, "benchmarks": [ { "benchmark_name": "Benchmark Name 1", "parameter_values": {"argument1": 1, "argument2": "value2"}, "value": 42, "val_min": 41.2, "val_q1": 41.5, "val_q3": 42.5, "val_max": 42.8, "std_dev": 0.5, "n_obs": 100, "run_metadata": {"any": "key-value pairs"}, "run_notes": "Any relevant notes." }, { "benchmark_name": "Benchmark Name 2", "parameter_values": {"not nullable": "Use {} if no params."}, "value": 8, "std_dev": 1, "n_obs": 2, } ] } - The entry for "context" contains the machine, environment, and timestamp information common to all of the runs - The entry for "benchmark_version" maps benchmark names to their version strings. (Which can be a git hash, the entire code string, a number, or any other string of your choice.) - The entry for "benchmarks" is a list of benchmark run data for the given context and benchmark versions. The first example benchmark run entry contains all possible values, even nullable ones, and the second entry omits all nullable values. back to `Benchmark data model `_ ingest_machine_view ------------------- :code:`ingest_machine_view(from_jsonb jsonb)` The argument is a JSON object. NOTE: key names must be entirely lowercase, or the insert will fail. Extra key-value pairs are ignored. Example:: { "mac_address": "0a:00:2d:01:02:03", "machine_name": "Yet-Another-Machine-Name", "memory_bytes": 8589934592, "cpu_actual_frequency_hz": 2300000000, "os_name": "OSX", "architecture_name": "x86_64", "kernel_name": "18.2.0", "cpu_model_name": "Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz", "cpu_core_count": 2, "cpu_thread_count": 4, "cpu_frequency_max_hz": 2300000000, "cpu_frequency_min_hz": 2300000000, "cpu_l1d_cache_bytes": 32768, "cpu_l1i_cache_bytes": 32768, "cpu_l2_cache_bytes": 262144, "cpu_l3_cache_bytes": 4194304, "machine_other_attributes": {"just": "an example"}, "gpu_information": "", "gpu_part_number": "", "gpu_product_name": "" } To identify which columns in "machine_view" are required, please see the view documentation in :ref:`benchmark-data-model`. back to `Benchmark data model `_ .. _benchmark-views: Benchmark views =============== benchmark_run_view ------------------ Each benchmark run. - Each entry is unique on the machine, environment, benchmark, and git commit timestamp. =============================== =========== ======== =========== =========== Column Type Nullable Default Description =============================== =========== ======== =========== =========== benchmark_run_id int8 not null serial primary key benchmark_name citext not null unique benchmark_version citext not null unique parameter_values jsonb not null '{}'::jsonb unique value numeric not null git_commit_timestamp timestamptz not null unique git_hash text not null val_min numeric val_q1 numeric val_q3 numeric val_max numeric std_dev numeric not null n_obs int4 not null run_timestamp timestamptz not null unique run_metadata jsonb run_notes text mac_address macaddr not null unique benchmark_language citext not null unique language_implementation_version citext not null ''::citext unique dependencies jsonb not null '{}'::jsonb unique =============================== =========== ======== =========== =========== back to `Benchmark data model `_ benchmark_view -------------- The details about a particular benchmark. - "benchmark_name" is unique for a given "benchmark_language" - Each entry is unique on ("benchmark_language", "benchmark_name", "benchmark_version") ===================== ====== ======== ======= =========== Column Type Nullable Default Description ===================== ====== ======== ======= =========== benchmark_id int4 not null serial primary key benchmark_name citext not null unique parameter_names _text benchmark_description text not null benchmark_type citext not null unique units citext not null unique lessisbetter bool not null benchmark_version citext not null unique benchmark_language citext not null unique ===================== ====== ======== ======= =========== back to `Benchmark data model `_ environment_view ---------------- The build environment used for a reported benchmark run. (Will be inferred from each "benchmark_run" if not explicitly added). - Each entry is unique on ("benchmark_language", "language_implementation_version", "dependencies") - "benchmark_language" is unique in the "benchmark_language" table - "benchmark_language" plus "language_implementation_version" is unique in the "language_implementation_version" table - "dependencies" is unique in the "dependencies" table =============================== ====== ======== =========== =========== Column Type Nullable Default Description =============================== ====== ======== =========== =========== environment_id int4 not null serial primary key benchmark_language citext not null unique language_implementation_version citext not null ''::citext unique dependencies jsonb not null '{}'::jsonb unique =============================== ====== ======== =========== =========== back to `Benchmark data model `_ machine_view ------------ The machine environment (CPU, GPU, OS) used for each benchmark run. - "mac_address" is unique in the "machine" table - "gpu_part_number" is unique in the "gpu" (graphics processing unit) table Empty string (''), not null, is used for machines that won't use the GPU - "cpu_model_name" is unique in the "cpu" (central processing unit) table - "os_name", "os_architecture_name", and "os_kernel_name" are unique in the "os" (operating system) table - "machine_other_attributes" is a key-value store for any other relevant data, e.g. '{"hard_disk_type": "solid state"}' ======================== ======= ======== ========== =========== Column Type Nullable Default Description ======================== ======= ======== ========== =========== machine_id int4 not null serial primary key mac_address macaddr not null unique machine_name citext not null memory_bytes int8 not null cpu_actual_frequency_hz int8 not null os_name citext not null unique architecture_name citext not null unique kernel_name citext not null ''::citext unique cpu_model_name citext not null unique cpu_core_count int4 not null cpu_thread_count int4 not null cpu_frequency_max_hz int8 not null cpu_frequency_min_hz int8 not null cpu_l1d_cache_bytes int4 not null cpu_l1i_cache_bytes int4 not null cpu_l2_cache_bytes int4 not null cpu_l3_cache_bytes int4 not null gpu_information citext not null ''::citext unique gpu_part_number citext not null ''::citext gpu_product_name citext not null ''::citext machine_other_attributes jsonb ======================== ======= ======== ========== =========== back to `Benchmark data model `_