diff options
Diffstat (limited to '')
| -rw-r--r-- | plugin/handler_socket/docs-en/about-handlersocket.en.txt | 72 | ||||
| -rw-r--r-- | plugin/handler_socket/docs-en/configuration-options.en.txt | 99 | ||||
| -rw-r--r-- | plugin/handler_socket/docs-en/installation.en.txt | 92 | ||||
| -rw-r--r-- | plugin/handler_socket/docs-en/perl-client.en.txt | 134 | ||||
| -rw-r--r-- | plugin/handler_socket/docs-en/protocol.en.txt | 205 |
5 files changed, 602 insertions, 0 deletions
diff --git a/plugin/handler_socket/docs-en/about-handlersocket.en.txt b/plugin/handler_socket/docs-en/about-handlersocket.en.txt new file mode 100644 index 00000000..73b9cab5 --- /dev/null +++ b/plugin/handler_socket/docs-en/about-handlersocket.en.txt @@ -0,0 +1,72 @@ + +----------------------------------------------------------------------------- +HandlerSocket plugin for MySQL + +Copyright (c) 2010 DeNA Co.,Ltd. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of DeNA Co.,Ltd. nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY DeNA Co.,Ltd. "AS IS" AND ANY EXPRESS OR +IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO +EVENT SHALL DeNA Co.,Ltd. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; +OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR +OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF +ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +----------------------------------------------------------------------------- +About HandlerSocket + +HandlerSocket is a NoSQL plugin for MySQL. It works as a daemon inside the +mysqld process, accept tcp connections, and execute requests from clients. +HandlerSocket does not support SQL queries. Instead, it supports simple CRUD +operations on tables. + +Because of the following reasons, HandlerSocket is much faster than the +mysqld/libmysql pair in some circumstances: + + - HandlerSocket manipulates data without parsing SQL, which causes less + CPU usage. + - HandlerSocket reads many requests from clients and executes their + requests in bulk, which causes less CPU and disk usage. + - HandlerSocket client/server protocol is more compact than the + mysql/libmysql pair, which causes less network usage. + +The current version of HandlerSocket only works with GNU/Linux. The source +archive of HandlerSocket includes a C++ and a Perl client libraries. +Here is a list of other language bindings: + + - PHP + http://openpear.org/package/Net_HandlerSocket + http://github.com/tz-lom/HSPHP + http://code.google.com/p/php-handlersocket/ + - Java + http://code.google.com/p/handlersocketforjava/ + - Python + https://code.launchpad.net/~songofacandy/+junk/pyhandlersocket + - Ruby + https://github.com/winebarrel/ruby-handlersocket + https://github.com/miyucy/handlersocket + - JavaScript(Node.js) + https://github.com/koichik/node-handlersocket + +The home of HandlerSocket is here: + https://github.com/DeNADev/HandlerSocket-Plugin-for-MySQL + +More documents are available in docs-en/ and docs-ja/ directories. + diff --git a/plugin/handler_socket/docs-en/configuration-options.en.txt b/plugin/handler_socket/docs-en/configuration-options.en.txt new file mode 100644 index 00000000..60fb6d85 --- /dev/null +++ b/plugin/handler_socket/docs-en/configuration-options.en.txt @@ -0,0 +1,99 @@ + +----------------------------------------------------------------- +handlersocket_verbose (default = 10, min = 0, max = 10000) + + Specify the logging verboseness. + +----------------------------------------------------------------- +handlersocket_address (default = '') + + Specify the address to bind. If empty, it binds to 0.0.0.0. + +----------------------------------------------------------------- +handlersocket_port (default = '9998') + + Specify the port to bind. This option is for the listener for + read requests. If empty, the listener is disabled. + +----------------------------------------------------------------- +handlersocket_port_wr (default = '9999') + + Specify the port to bind. This option is for the listener for + write requests. If empty, the listener is disabled. + +----------------------------------------------------------------- +handlersocket_epoll (default = 1, min = 0, max = 1) + + Specify whether handlersocket uses epoll for I/O multiplexing. + +----------------------------------------------------------------- +handlersocket_threads (default = 16, min = 1, max = 3000) + + Specify the number of handlersocket worker threads. This option + is for the listener for read requests. Recommended value is + (the number of CPU cores * 2). + +----------------------------------------------------------------- +handlersocket_threads_wr (default = 1, min = 1, max = 3000) + + Specify the number of handlersocket worker threads. This option + is for the listener for write requests. Recommended value is 1. + +----------------------------------------------------------------- +handlersocket_timeout (default = 300, min = 30, max = 3600) + + Specify the socket timeout in seconds. + +----------------------------------------------------------------- +handlersocket_backlog (default = 32768, min = 5, max = 1000000) + + Specify the length of the listen backlog. + +----------------------------------------------------------------- +handlersocket_sndbuf (default = 0, min = 0, max = 1677216) + + Specify the maximum socket send buffer in bytes. If 0, the + system-wide default value is set. + +----------------------------------------------------------------- +handlersocket_rcvbuf (default = 0, min = 0, max = 1677216) + + Specify the maximum socket receive buffer in bytes. If 0, the + system-wide default value is set. + +----------------------------------------------------------------- +handlersocket_readsize (default = 0, min = 0, max = 1677216) + + Specify the minimum length of the handlersocket request buffer. + Larger value can make handlersocket faster for large requests, + but can consume memory. The default value is possibly 4096. + +----------------------------------------------------------------- +handlersocket_accept_balance (default = 0, min = 0, max = 10000) + + When this option is set to non-zero, handlersocket tries to + balance accepted connections among threads. Non-zero is + recommended if you use persistent connections (i.e., connection + pooling on the client side). + +----------------------------------------------------------------- +handlersocket_wrlock_timeout (default = 12, min = 0, max = 3600) + + Specify the lock timeout in seconds. When a write request is + performed, handlersocket acquires an advisory lock named + 'handlersocket_wr'. This option sets the timeout for the + locking. + +----------------------------------------------------------------- +handlersocket_plain_secret (default = '') + + When this option is specified, a plain-text authentication is + enabled for the listener for read requests. This option + specifies the secret key for the authentication. + +----------------------------------------------------------------- +handlersocket_plain_secret_wr (default = '') + + This option specifies the secret key for the listener for write + requests. + diff --git a/plugin/handler_socket/docs-en/installation.en.txt b/plugin/handler_socket/docs-en/installation.en.txt new file mode 100644 index 00000000..0a0616fa --- /dev/null +++ b/plugin/handler_socket/docs-en/installation.en.txt @@ -0,0 +1,92 @@ +1. Building Handlersocket + + Handlersocket mainly consists of libhsclient, handlersocket, and C++/Perl clients. libhsclient is a common library shared from both client and server(plugin). handlersocket is a MySQL daemon plugin. + To build Handlersocket, you need both MySQL source code and MySQL binary. It is not required to pre-build MySQL source code, but source itself is needed because Handlersocket depends on MySQL header files that only MySQL source distribution contains. MySQL binary is just a normal MySQL binary distribution. You can use official MySQL binaries provided by Oracle. + Since Handlersocket uses daemon plugin interface supported from MySQL 5.1, +MySQL 5.1 or higher version is required. + Please make sure that you use identical MySQL version between MySQL source +and MySQL binary. Otherwise you might encounter serious problems (i.e. server +crash, etc). + Here are steps to build Handlersocket. + +* Get MySQL source code + +* Get MySQL binary + +* Build Handlersocket + $ ./autogen.sh + $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin + + --with-mysql-source refers to the top of MySQL source directory (which +contains the VERSION file or the configure.in file), --with-mysql-bindir +refers to where MySQL binary executables (i.e. mysql_config) are located, +and --with-mysql-plugindir refers to a plugin directory where plugin +libraries (*.so) are installed. + + $ make + $ sudo make install + + Both libhsclient and the handlersocket plugin will be installed. + + +2. Using Handlersocket + +Append configuration options for handlersocket to my.cnf. + + [mysqld] + loose_handlersocket_port = 9998 + # the port number to bind to (for read requests) + loose_handlersocket_port_wr = 9999 + # the port number to bind to (for write requests) + loose_handlersocket_threads = 16 + # the number of worker threads (for read requests) + loose_handlersocket_threads_wr = 1 + # the number of worker threads (for write requests) + open_files_limit = 65535 + # to allow handlersocket accept many concurrent + # connections, make open_files_limit as large as + # possible. + +Log in to mysql as root, and execute the following query. + + mysql> install plugin handlersocket soname 'handlersocket.so'; + +If handlersocket.so is successfully installed, it starts +accepting connections on port 9998 and 9999. Running +'show processlist' should show handlersocket worker threads. + +----------------------------------------------------------------- +On the client side, you need to install libhsclient for c++ apps +and perl-Net-HandlerSocket for perl apps. They do not require +MySQL to compile. + + $ ./autogen.sh + $ ./configure --disable-handlersocket-server + $ make + $ sudo make install + $ cd perl-Net-HandlerSocket + $ perl Makefile.PL + $ make + $ sudo make install + +----------------------------------------------------------------- +Alternatively, you can use the rpm installation. If your OS +supports rpms, you can use the following commands to build and +install handlersocket rpm packages. + +(Server side, installs HandlerSocket plugin) + $ ./autogen.sh + $ ./configure --with-mysql-source=/work/mysql-5.1.50 --with-mysql-bindir=/work/mysql-5.1.50-linux-x86_64-glibc23/bin --with-mysql-plugindir=/work/mysql-5.1.50-linux-x86_64-glibc23/lib/plugin + $ make rpm_cli + $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm + $ make rpm_c + $ sudo rpm -U dist/RPMS/*/handlersocket*.rpm + +(Client side, installs client libraries) + $ ./autogen.sh + $ ./configure --disable-handlersocket-server + $ make rpm_cli + $ sudo rpm -U dist/RPMS/*/libhsclient*.rpm + $ make rpm_perl + $ sudo rpm -U dist/RPMS/*/perl-Net-HandlerSocket*.rpm + diff --git a/plugin/handler_socket/docs-en/perl-client.en.txt b/plugin/handler_socket/docs-en/perl-client.en.txt new file mode 100644 index 00000000..cc913851 --- /dev/null +++ b/plugin/handler_socket/docs-en/perl-client.en.txt @@ -0,0 +1,134 @@ + +----------------------------------------------------------------- +To open a connection to the handlersocket plugin, you need to +create a Net::HandlerSocket object. + + use Net::HandlerSocket; + my $args = { host => 'localhost', port => 9998 }; + my $hs = new Net::HandlerSocket($args); + +----------------------------------------------------------------- +Before executing table operations, you need to open an index to +work with. + + my $err = $hs->open_index(3, 'database1', 'table1', 'PRIMARY', + 'f1,f2'); + die $hs->get_error() if $res->[0] != 0; + +The first argument for open_index is an integer value which is +used to identify an open table, which is only valid within the +same Net::HandlerSocket object. The 4th argument is the name of +index to open. If 'PRIMARY' is specified, the primary index is +open. The 5th argument is a comma-separated list of column names. + +----------------------------------------------------------------- +To read a record from a table using an index, call the +execute_single method. + + my $res = $hs->execute_single(3, '=', [ 'foo' ], 1, 0); + die $hs->get_error() if $res->[0] != 0; + shift(@$res); + +The first argument must be an integer which has specified as the +first argument for open_index on the same Net::HandlerSocket +object. The second argument specifies the search operation. The +current version of handlersocket supports '=', '>=', '<=', '>', +and '<'. The 3rd argument specifies the key to find, which must +an arrayref whose length is equal to or smaller than the number +of key columns of the index. The 4th and the 5th arguments +specify the maximum number of records to be retrieved, and the +number of records skipped before retrieving records. The columns +to be retrieved are specified by the 5th argument for the +corresponding open_index call. + +The execute_single method always returns an arrayref. The first +element is the error code, which is 0 when no error is occurred. +The remaining are the field values. If more than one record is +returned, it is flatten to an 1-dimensional array. For example, +when 5 records that have 3 columns are returned, you can retrieve +values using the following code. + + die $hs->get_error() if $res->[0] != 0; + shift(@$res); + for (my $row = 0; $row < 5; ++$row) { + for (my $col = 0; $col < 3; ++$col) { + my $value = $res->[$row * 5 + $col]; + # ... + } + } + +----------------------------------------------------------------- +To update or delete records, you need to specify more arguments +for the execute_single method. Note that the Net::HandlerSocket +object must be connected to a handlersocket worker for write +operations, which is port 9999 by default. +(For safety, the port 9998 only allows read operations, and the +port 9999 allows write operations also. The port 9999 allows +read operations too, but slower than 9998 because of record +locking etc.. Port numbers can be changed using the +'handlersocket_port' and the 'handlersocket_port_wr' +configuration options of mysqld.) + + my $args = { host => 'localhost', port => 9999 }; + my $hs = new Net::HandlerSocket($args); + + my $res = $hs->execute_single(3, '=', [ 'bar' ], 1, 0, 'U', + [ 'fubar', 'hoge' ]); + die $hs->get_error() if $res->[0] != 0; + my $num_updated_rows = $res->[1]; + + my $res = $hs->execute_single(3, '=', [ 'baz' ], 1, 0, 'D'); + die $hs->get_error() if $res->[0] != 0; + my $num_deleted_rows = $res->[1]; + +The 6th argument for execute_single specifies the modification +operation. The current version supports 'U' and 'D'. For the 'U' +operation, the 7th argument specifies the new value for the row. +The columns to be modified are specified by the 5th argument for +the corresponding open_index call. For the 'D' operation, the +7th argument can be omitted. + +----------------------------------------------------------------- +The execute_single method can be used for inserting records also. + + my $res = $hs->execute_single(3, '+', [ 'foo', 'bar', 'baz' ]); + die $hs->get_error() if $res->[0] != 0; + +The 3rd argument must be an arrayref whose elements correspond to +the 5th argument for the corresponding open_index call. If there +is a column which is not appeared in the 5th argument for the +open_index, the default value for the column is set. + +----------------------------------------------------------------- +Multiple operations can be executed in a single call. Executing +multiple operations in a single call is much faster than +executing them separatedly. + + my $rarr = $hs->execute_multi([ + [ 0, '>=', [ 'foo' ], 5, 0 ], + [ 2, '=', [ 'bar' ], 1, 0 ], + [ 4, '<', [ 'baz' ], 10, 5 ], + ]); + for my $res (@$rarr) { + die $hs->get_error() if $res->[0] != 0; + shift(@$res); + # ... + } + +----------------------------------------------------------------- +If handlersocket is configured to authenticate client connections +(ie., handlersocket_plain_secret or handlersocket_plain_secret_wr +is set), a client must call 'auth' method before any other +methods. + + my $res = $hs->auth('password'); + die $hs->get_error() if $res->[0] != 0; + +----------------------------------------------------------------- +When an error is occurred, the first element of the returned +arrayref becomes a non-zero value. A negative value indicates +that an I/O error is occurred and the Net::HandlerSocket object +should be disposed. A positive value means that the connection is +still active and the Net::HandlerSocket object can be reused +later. + diff --git a/plugin/handler_socket/docs-en/protocol.en.txt b/plugin/handler_socket/docs-en/protocol.en.txt new file mode 100644 index 00000000..3518be36 --- /dev/null +++ b/plugin/handler_socket/docs-en/protocol.en.txt @@ -0,0 +1,205 @@ + +---------------------------------------------------------------------------- +The HandlerSocket protocol + +---------------------------------------------------------------------------- +Basic syntax + +- The HandlerSocket protocol is line-based. Each line ends with LF(0x0a). +- Each line consists a concatenation of tokens separated by HT(0x09). +- A token is either NULL or an encoded string. Note that you need to + distinguish NULL from an empty string, as most DBMs does so. +- NULL is expressed as a single NUL(0x00). +- An encoded string is a string with the following encoding rules. + - Characters in the range [0x10 - 0xff] are encoded as itselves. + - A character in the range [0x00 - 0x0f] is prefixed by 0x01 and + shifted by 0x40. For example, 0x03 is encoded as 0x01 0x43. +- Note that a string can be empty. A continuation of 0x09 0x09 means that + there is an empty string between them. A continuation of 0x09 0x0a means + that there is an empty string at the end of the line. + +---------------------------------------------------------------------------- +Request and Response + +- The HandlerSocket protocol is a simple request/response protocol. After a + connection is established, the client side sends a request, and then the + server side sends a response. +- A request/response consists of a single line. +- Requests can be pipelined; That is, you can send multiple requests (ie. + lines) at one time, and receive responses for them at one time. + +---------------------------------------------------------------------------- +Opening index + +The 'open_index' request has the following syntax. + + P <indexid> <dbname> <tablename> <indexname> <columns> [<fcolumns>] + +- <indexid> is a number in decimal. +- <dbname>, <tablename>, and <indexname> are strings. To open the primary + key, use PRIMARY as <indexname>. +- <columns> is a comma-separated list of column names. +- <fcolumns> is a comma-separated list of column names. This parameter is + optional. + +Once an 'open_index' request is issued, the HandlerSocket plugin opens the +specified index and keep it open until the client connection is closed. Each +open index is identified by <indexid>. If <indexid> is already open, the old +open index is closed. You can open the same combination of <dbname> +<tablename> <indexname> multiple times, possibly with different <columns>. +For efficiency, keep <indexid> small as far as possible. + +---------------------------------------------------------------------------- +Getting data + +The 'find' request has the following syntax. + + <indexid> <op> <vlen> <v1> ... <vn> [LIM] [IN] [FILTER ...] + +LIM is a sequence of the following parameters. + + <limit> <offset> + +IN is a sequence of the following parameters. + + @ <icol> <ivlen> <iv1> ... <ivn> + +FILETER is a sequence of the following parameters. + + <ftyp> <fop> <fcol> <fval> + +- <indexid> is a number. This number must be an <indexid> specified by a + 'open_index' request executed previously on the same connection. +- <op> specifies the comparison operation to use. The current version of + HandlerSocket supports '=', '>', '>=', '<', and '<='. +- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This + must be smaller than or equal to the number of index columns specified by + the <indexname> parameter of the corresponding 'open_index' request. +- <v1> ... <vn> specify the index column values to fetch. +- LIM is optional. <limit> and <offset> are numbers. When omitted, it works + as if 1 and 0 are specified. These parameter works like LIMIT of SQL. + These values don't include the number of records skipped by a filter. +- IN is optional. It works like WHERE ... IN syntax of SQL. <icol> must be + smaller than the number of index columns specified by the <indexname> + parameter of the corresponding 'open_index' request. If IN is specified in + a find request, the <icol>-th parameter value of <v1> ... <vn> is ignored. +- FILTERs are optional. A FILTER specifies a filter. <ftyp> is either 'F' + (filter) or 'W' (while). <fop> specifies the comparison operation to use. + <fcol> must be smaller than the number of columns specified by the + <fcolumns> parameter of the corresponding 'open_index' request. Multiple + filters can be specified, and work as the logical AND of them. The + difference of 'F' and 'W' is that, when a record does not meet the + specified condition, 'F' simply skips the record, and 'W' stops the loop. + +---------------------------------------------------------------------------- +Updating/Deleting data + +The 'find_modify' request has the following syntax. + + <indexid> <op> <vlen> <v1> ... <vn> [LIM] [IN] [FILTER ...] MOD + +MOD is a sequence of the following parameters. + + <mop> <m1> ... <mk> + +- <mop> is 'U' (update), '+' (increment), '-' (decrement), 'D' (delete), + 'U?', '+?', '-?', or 'D?'. If the '?' suffix is specified, it returns + the contents of the records before modification (as if it's a 'find' + request), instead of the number of modified records. +- <m1> ... <mk> specifies the column values to set. The length of <m1> ... + <mk> must be smaller than or equal to the length of <columns> specified by + the corresponding 'open_index' request. If <mop> is 'D', these parameters + are ignored. If <mop> is '+' or '-', values must be numeric. If <mop> is + '-' and it attempts to change a column value from negative to positive or + positive to negative, the column value is not modified. + +---------------------------------------------------------------------------- +Inserting data + +The 'insert' request has the following syntax. + + <indexid> + <vlen> <v1> ... <vn> + +- <vlen> indicates the length of the trailing parameters <v1> ... <vn>. This + must be smaller than or equal to the length of <columns> specified by the + corresponding 'open_index' request. +- <v1> ... <vn> specify the column values to set. For columns not in + <columns>, the default values for each column are set. + +---------------------------------------------------------------------------- +Authentication + +The 'auth' request has the following syntax. + + A <atyp> <akey> + +- <atyp> must be '1' +- An 'auth' request succeeds iff <akey> is the correct secret specified by + the 'handlersocket_plain_secret' or 'handlersocket_plain_secret_rw'. +- If an authentication is enabled for a listener, any other requests on a + connection fail before an 'auth' request succeeded on the connection. + +---------------------------------------------------------------------------- +Response syntax + +HandlerSocket returns a response of the following syntax for each request. + + <errorcode> <numcolumns> <r1> ... <rn> + +- <errorcode> indicates whether the request has successfully executed or not. + '0' means success. Non-zero means an error. +- <numcolumns> indicates the number of columns of the result set. +- <r1> ... <rn> is the result set. The length of <r1> ... <rn> is always a + multiple of <numcolumns>. It is possible that <r1> ... <rn> is empty. + +If <errorcode> is non-zero, <numcolumns> is always 1 and <r1> indicates a +human-readable error message, though sometimes <r1> is not provided. + +---------------------------------------------------------------------------- +Response for 'open_index' + +If 'open_index' is succeeded, HandlerSocket returns a line of the following +syntax. + + 0 1 + +---------------------------------------------------------------------------- +Response for 'find' + +If 'find' is succeeded, HandlerSocket returns a line of the following +syntax. + + 0 <numcolumns> <r1> ... <rn> + +- <numcolumns> always equals to the length of <columns> of the corresponding + 'open_index' request. +- <r1> ... <rn> is the result set. If N rows are found, the length of <r1> + ... <rn> becomes ( <numcolumns> * N ). + +---------------------------------------------------------------------------- +Response for 'find_modify' + +If 'find_modify' is succeeded, HandlerSocket returns a line of the following +syntax. + + 0 1 <nummod> + +- <nummod> is the number of modified rows. +- As an exception, if the '?' suffix is specified in <mop>, a response has + the syntax of a response for 'find' instead. + +---------------------------------------------------------------------------- +Response for 'insert' + +If 'insert' is succeeded, HanderSocket returns a line of the following +syntax. + + 0 1 + +---------------------------------------------------------------------------- +Response for 'auth' + +If 'auth' is succeeded, HanderSocket returns a line of the following syntax. + + 0 1 + |