Mercurial > hg > nginx-site
view xml/en/docs/dev/development_guide.xml @ 2797:17ed19394953
Documented the mp4_start_key_frame directive.
author | Yaroslav Zhuravlev <yar@nginx.com> |
---|---|
date | Mon, 15 Nov 2021 12:52:47 +0000 |
parents | 5e3591372bf6 |
children | b7ff3d1915a1 |
line wrap: on
line source
<?xml version="1.0"?> <!-- Copyright (C) Nginx, Inc. --> <!DOCTYPE article SYSTEM "../../../../dtd/article.dtd"> <article name="Development guide" link="/en/docs/dev/development_guide.html" lang="en" rev="10"> <section name="Introduction" id="introduction"> <section name="Code layout" id="code_layout"> <list type="bullet"> <listitem> <literal>auto</literal> — Build scripts </listitem> <listitem> <literal>src</literal> <list type="bullet"> <listitem> <literal>core</literal> — Basic types and functions — string, array, log, pool, etc. </listitem> <listitem> <literal>event</literal> — Event core <list type="bullet"> <listitem> <literal>modules</literal> — Event notification modules: <literal>epoll</literal>, <literal>kqueue</literal>, <literal>select</literal> etc. </listitem> </list> </listitem> <listitem> <literal>http</literal> — Core HTTP module and common code <list type="bullet"> <listitem> <literal>modules</literal> — Other HTTP modules </listitem> <listitem> <literal>v2</literal> — HTTP/2 </listitem> </list> </listitem> <listitem> <literal>mail</literal> — Mail modules </listitem> <listitem> <literal>os</literal> — Platform-specific code <list type="bullet"> <listitem> <literal>unix</literal> </listitem> <listitem> <literal>win32</literal> </listitem> </list> </listitem> <listitem> <literal>stream</literal> — Stream modules </listitem> </list> </listitem> </list> </section> <section name="Include files" id="include_files"> <para> The following two <literal>#include</literal> statements must appear at the beginning of every nginx file: </para> <programlisting> #include <ngx_config.h> #include <ngx_core.h> </programlisting> <para> In addition to that, HTTP code should include </para> <programlisting> #include <ngx_http.h> </programlisting> <para> Mail code should include </para> <programlisting> #include <ngx_mail.h> </programlisting> <para> Stream code should include </para> <programlisting> #include <ngx_stream.h> </programlisting> </section> <section name="Integers" id="integers"> <para> For general purposes, nginx code uses two integer types, <literal>ngx_int_t</literal> and <literal>ngx_uint_t</literal>, which are typedefs for <literal>intptr_t</literal> and <literal>uintptr_t</literal> respectively. </para> </section> <section name="Common return codes" id="common_return_codes"> <para> Most functions in nginx return the following codes: </para> <list type="bullet"> <listitem> <literal>NGX_OK</literal> — Operation succeeded. </listitem> <listitem> <literal>NGX_ERROR</literal> — Operation failed. </listitem> <listitem> <literal>NGX_AGAIN</literal> — Operation incomplete; call the function again. </listitem> <listitem> <literal>NGX_DECLINED</literal> — Operation rejected, for example, because it is disabled in the configuration. This is never an error. </listitem> <listitem> <literal>NGX_BUSY</literal> — Resource is not available. </listitem> <listitem> <literal>NGX_DONE</literal> — Operation complete or continued elsewhere. Also used as an alternative success code. </listitem> <listitem> <literal>NGX_ABORT</literal> — Function was aborted. Also used as an alternative error code. </listitem> </list> </section> <section name="Error handling" id="error_handling"> <para> The <literal>ngx_errno</literal> macro returns the last system error code. It's mapped to <literal>errno</literal> on POSIX platforms and to <literal>GetLastError()</literal> call in Windows. The <literal>ngx_socket_errno</literal> macro returns the last socket error number. Like the <literal>ngx_errno</literal> macro, it's mapped to <literal>errno</literal> on POSIX platforms. It's mapped to the <literal>WSAGetLastError()</literal> call on Windows. Accessing the values of <literal>ngx_errno</literal> or <literal>ngx_socket_errno</literal> more than once in a row can cause performance issues. If the error value might be used multiple times, store it in a local variable of type <literal>ngx_err_t</literal>. To set errors, use the <literal>ngx_set_errno(errno)</literal> and <literal>ngx_set_socket_errno(errno)</literal> macros. </para> <para> The values of <literal>ngx_errno</literal> and <literal>ngx_socket_errno</literal> can be passed to the logging functions <literal>ngx_log_error()</literal> and <literal>ngx_log_debugX()</literal>, in which case system error text is added to the log message. </para> <para> Example using <literal>ngx_errno</literal>: </para> <programlisting> ngx_int_t ngx_my_kill(ngx_pid_t pid, ngx_log_t *log, int signo) { ngx_err_t err; if (kill(pid, signo) == -1) { err = ngx_errno; ngx_log_error(NGX_LOG_ALERT, log, err, "kill(%P, %d) failed", pid, signo); if (err == NGX_ESRCH) { return 2; } return 1; } return 0; } </programlisting> </section> </section> <section name="Strings" id="strings"> <section name="Overview" id="string_overview"> <para> For C strings, nginx uses the unsigned character type pointer <literal>u_char *</literal>. </para> <para> The nginx string type <literal>ngx_str_t</literal> is defined as follows: </para> <programlisting> typedef struct { size_t len; u_char *data; } ngx_str_t; </programlisting> <para> The <literal>len</literal> field holds the string length and <literal>data</literal> holds the string data. The string, held in <literal>ngx_str_t</literal>, may or may not be null-terminated after the <literal>len</literal> bytes. In most cases it’s not. However, in certain parts of the code (for example, when parsing configuration), <literal>ngx_str_t</literal> objects are known to be null-terminated, which simplifies string comparison and makes it easier to pass the strings to syscalls. </para> <para> The string operations in nginx are declared in <path>src/core/ngx_string.h</path> Some of them are wrappers around standard C functions: </para> <para> <list type="bullet"> <listitem> <literal>ngx_strcmp()</literal> </listitem> <listitem> <literal>ngx_strncmp()</literal> </listitem> <listitem> <literal>ngx_strstr()</literal> </listitem> <listitem> <literal>ngx_strlen()</literal> </listitem> <listitem> <literal>ngx_strchr()</literal> </listitem> <listitem> <literal>ngx_memcmp()</literal> </listitem> <listitem> <literal>ngx_memset()</literal> </listitem> <listitem> <literal>ngx_memcpy()</literal> </listitem> <listitem> <literal>ngx_memmove()</literal> </listitem> </list> </para> <para> Other string functions are nginx-specific </para> <para> <list type="bullet"> <listitem> <literal>ngx_memzero()</literal> — Fills memory with zeroes. </listitem> <listitem> <literal>ngx_explicit_memzero()</literal> — Does the same as <literal>ngx_memzero()</literal>, but this call is never removed by the compiler's dead store elimination optimization. This function can be used to clear sensitive data such as passwords and keys. </listitem> <listitem> <literal>ngx_cpymem()</literal> — Does the same as <literal>ngx_memcpy()</literal>, but returns the final destination address This one is handy for appending multiple strings in a row. </listitem> <listitem> <literal>ngx_movemem()</literal> — Does the same as <literal>ngx_memmove()</literal>, but returns the final destination address. </listitem> <listitem> <literal>ngx_strlchr()</literal> — Searches for a character in a string, delimited by two pointers. </listitem> </list> </para> <para> The following functions perform case conversion and comparison: </para> <para> <list type="bullet"> <listitem> <literal>ngx_tolower()</literal> </listitem> <listitem> <literal>ngx_toupper()</literal> </listitem> <listitem> <literal>ngx_strlow()</literal> </listitem> <listitem> <literal>ngx_strcasecmp()</literal> </listitem> <listitem> <literal>ngx_strncasecmp()</literal> </listitem> </list> </para> <para> The following macros simplify string initialization: </para> <list type="bullet"> <listitem> <literal>ngx_string(text)</literal> — static initializer for the <literal>ngx_str_t</literal> type from the C string literal <literal>text</literal> </listitem> <listitem> <literal>ngx_null_string</literal> — static empty string initializer for the <literal>ngx_str_t</literal> type </listitem> <listitem> <literal>ngx_str_set(str, text)</literal> — initializes string <literal>str</literal> of <literal>ngx_str_t *</literal> type with the C string literal <literal>text</literal> </listitem> <listitem> <literal>ngx_str_null(str)</literal> — initializes string <literal>str</literal> of <literal>ngx_str_t *</literal> type with the empty string </listitem> </list> </section> <section name="Formatting" id="formatting"> <para> The following formatting functions support nginx-specific types: </para> <para> <list type="bullet"> <listitem> <literal>ngx_sprintf(buf, fmt, ...)</literal> </listitem> <listitem> <literal>ngx_snprintf(buf, max, fmt, ...)</literal> </listitem> <listitem> <literal>ngx_slprintf(buf, last, fmt, ...)</literal> </listitem> <listitem> <literal>ngx_vslprintf(buf, last, fmt, args)</literal> </listitem> <listitem> <literal>ngx_vsnprintf(buf, max, fmt, args)</literal> </listitem> </list> </para> <para> The full list of formatting options, supported by these functions is in <path>src/core/ngx_string.c</path>. Some of them are: </para> <list type="bullet"> <listitem> <literal>%O</literal> — <literal>off_t</literal> </listitem> <listitem> <literal>%T</literal> — <literal>time_t</literal> </listitem> <listitem> <literal>%z</literal> — <literal>ssize_t</literal> </listitem> <listitem> <literal>%i</literal> — <literal>ngx_int_t</literal> </listitem> <listitem> <literal>%p</literal> — <literal>void *</literal> </listitem> <listitem> <literal>%V</literal> — <literal>ngx_str_t *</literal> </listitem> <listitem> <literal>%s</literal> — <literal>u_char *</literal> (null-terminated) </listitem> <listitem> <literal>%*s</literal> — <literal>size_t + u_char *</literal> </listitem> </list> <para> You can prepend <literal>u</literal> on most types to make them unsigned. To convert output to hex, use <literal>X</literal> or <literal>x</literal>. </para> <para> For example: <programlisting> u_char buf[NGX_INT_T_LEN]; size_t len; ngx_uint_t n; /* set n here */ len = ngx_sprintf(buf, "%ui", n) — buf; </programlisting> </para> </section> <section name="Numeric conversion" id="numeric_conversion"> <para> Several functions for numeric conversion are implemented in nginx. The first four each convert a string of given length to a positive integer of the indicated type. They return <literal>NGX_ERROR</literal> on error. <list type="bullet"> <listitem> <literal>ngx_atoi(line, n)</literal> — <literal>ngx_int_t</literal> </listitem> <listitem> <literal>ngx_atosz(line, n)</literal> — <literal>ssize_t</literal> </listitem> <listitem> <literal>ngx_atoof(line, n)</literal> — <literal>off_t</literal> </listitem> <listitem> <literal>ngx_atotm(line, n)</literal> — <literal>time_t</literal> </listitem> </list> </para> <para> There are two additional numeric conversion functions. Like the first four, they return <literal>NGX_ERROR</literal> on error. <list type="bullet"> <listitem> <literal>ngx_atofp(line, n, point)</literal> — Converts a fixed point floating number of given length to a positive integer of type <literal>ngx_int_t</literal>. The result is shifted left by <literal>point</literal> decimal positions. The string representation of the number is expected to have no more than <literal>points</literal> fractional digits. For example, <literal>ngx_atofp("10.5", 4, 2)</literal> returns <literal>1050</literal>. </listitem> <listitem> <literal>ngx_hextoi(line, n)</literal> — Converts a hexadecimal representation of a positive integer to <literal>ngx_int_t</literal>. </listitem> </list> </para> </section> <section name="Regular expressions" id="regex"> <para> The regular expressions interface in nginx is a wrapper around the <link url="http://www.pcre.org">PCRE</link> library. The corresponding header file is <path>src/core/ngx_regex.h</path>. </para> <para> To use a regular expression for string matching, it first needs to be compiled, which is usually done at the configuration phase. Note that since PCRE support is optional, all code using the interface must be protected by the surrounding <literal>NGX_PCRE</literal> macro: <programlisting> #if (NGX_PCRE) ngx_regex_t *re; ngx_regex_compile_t rc; u_char errstr[NGX_MAX_CONF_ERRSTR]; ngx_str_t value = ngx_string("message (\\d\\d\\d).*Codeword is '(?<cw>\\w+)'"); ngx_memzero(&rc, sizeof(ngx_regex_compile_t)); rc.pattern = value; rc.pool = cf->pool; rc.err.len = NGX_MAX_CONF_ERRSTR; rc.err.data = errstr; /* rc.options are passed as is to pcre_compile() */ if (ngx_regex_compile(&rc) != NGX_OK) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "%V", &rc.err); return NGX_CONF_ERROR; } re = rc.regex; #endif </programlisting> After successful compilation, the <literal>captures</literal> and <literal>named_captures</literal> fields in the <literal>ngx_regex_compile_t</literal> structure contain the count of all captures and named captures, respectively, found in the regular expression. </para> <para> The compiled regular expression can then be used for matching against strings: <programlisting> ngx_int_t n; int captures[(1 + rc.captures) * 3]; ngx_str_t input = ngx_string("This is message 123. Codeword is 'foobar'."); n = ngx_regex_exec(re, &input, captures, (1 + rc.captures) * 3); if (n >= 0) { /* string matches expression */ } else if (n == NGX_REGEX_NO_MATCHED) { /* no match was found */ } else { /* some error */ ngx_log_error(NGX_LOG_ALERT, log, 0, ngx_regex_exec_n " failed: %i", n); } </programlisting> The arguments to <literal>ngx_regex_exec()</literal> are the compiled regular expression <literal>re</literal>, the string to match <literal>input</literal>, an optional array of integers to hold any <literal>captures</literal> that are found, and the array's <literal>size</literal>. The size of the <literal>captures</literal> array must be a multiple of three, as required by the <link url="http://www.pcre.org/original/doc/html/pcreapi.html">PCRE API</link>. In the example, the size is calculated from the total number of captures plus one for the matched string itself. </para> <para> If there are matches, captures can be accessed as follows: <programlisting> u_char *p; size_t size; ngx_str_t name, value; /* all captures */ for (i = 0; i < n * 2; i += 2) { value.data = input.data + captures[i]; value.len = captures[i + 1] — captures[i]; } /* accessing named captures */ size = rc.name_size; p = rc.names; for (i = 0; i < rc.named_captures; i++, p += size) { /* capture name */ name.data = &p[2]; name.len = ngx_strlen(name.data); n = 2 * ((p[0] << 8) + p[1]); /* captured value */ value.data = &input.data[captures[n]]; value.len = captures[n + 1] — captures[n]; } </programlisting> </para> <para> The <literal>ngx_regex_exec_array()</literal> function accepts the array of <literal>ngx_regex_elt_t</literal> elements (which are just compiled regular expressions with associated names), a string to match, and a log. The function applies expressions from the array to the string until either a match is found or no more expressions are left. The return value is <literal>NGX_OK</literal> when there is a match and <literal>NGX_DECLINED</literal> otherwise, or <literal>NGX_ERROR</literal> in case of error. </para> </section> </section> <section name="Time" id="time"> <para> The <literal>ngx_time_t</literal> structure represents time with three separate types for seconds, milliseconds, and the GMT offset: <programlisting> typedef struct { time_t sec; ngx_uint_t msec; ngx_int_t gmtoff; } ngx_time_t; </programlisting> The <literal>ngx_tm_t</literal> structure is an alias for <literal>struct tm</literal> on UNIX platforms and <literal>SYSTEMTIME</literal> on Windows. </para> <para> To obtain the current time, it is usually sufficient to access one of the available global variables, representing the cached time value in the desired format. </para> <para> The available string representations are: <list type="bullet"> <listitem> <literal>ngx_cached_err_log_time</literal> — Used in error log entries: <literal>"1970/09/28 12:00:00"</literal> </listitem> <listitem> <literal>ngx_cached_http_log_time</literal> — Used in HTTP access log entries: <literal>"28/Sep/1970:12:00:00 +0600"</literal> </listitem> <listitem> <literal>ngx_cached_syslog_time</literal> — Used in syslog entries: <literal>"Sep 28 12:00:00"</literal> </listitem> <listitem> <literal>ngx_cached_http_time</literal> — Used in HTTP headers: <literal>"Mon, 28 Sep 1970 06:00:00 GMT"</literal> </listitem> <listitem> <literal>ngx_cached_http_log_iso8601</literal> — The ISO 8601 standard format: <literal>"1970-09-28T12:00:00+06:00"</literal> </listitem> </list> </para> <para> The <literal>ngx_time()</literal> and <literal>ngx_timeofday()</literal> macros return the current time value in seconds and are the preferred way to access the cached time value. </para> <para> To obtain the time explicitly, use <literal>ngx_gettimeofday()</literal>, which updates its argument (pointer to <literal>struct timeval</literal>). The time is always updated when nginx returns to the event loop from system calls. To update the time immediately, call <literal>ngx_time_update()</literal>, or <literal>ngx_time_sigsafe_update()</literal> if updating the time in the signal handler context. </para> <para> The following functions convert <literal>time_t</literal> into the indicated broken-down time representation. The first function in each pair converts <literal>time_t</literal> to <literal>ngx_tm_t</literal> and the second (with the <literal>_libc_</literal> infix) to <literal>struct tm</literal>: <list type="bullet"> <listitem> <literal>ngx_gmtime(), ngx_libc_gmtime()</literal> — Time expressed as UTC </listitem> <listitem> <literal>ngx_localtime(), ngx_libc_localtime()</literal> — Time expressed relative to the local time zone </listitem> </list> The <literal>ngx_http_time(buf, time)</literal> function returns a string representation suitable for use in HTTP headers (for example, <literal>"Mon, 28 Sep 1970 06:00:00 GMT"</literal>). The <literal>ngx_http_cookie_time(buf, time)</literal> returns a string representation function returns a string representation suitable for HTTP cookies (<literal>"Thu, 31-Dec-37 23:55:55 GMT"</literal>). </para> </section> <section name="Containers" id="containers"> <section name="Array" id="array"> <para> The nginx array type <literal>ngx_array_t</literal> is defined as follows </para> <programlisting> typedef struct { void *elts; ngx_uint_t nelts; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_array_t; </programlisting> <para> The elements of the array are available in the <literal>elts</literal> field. The <literal>nelts</literal> field holds the number of elements. The <literal>size</literal> field holds the size of a single element and is set when the array is initialized. </para> <para> Use the <literal>ngx_array_create(pool, n, size)</literal> call to create an array in a pool, and the <literal>ngx_array_init(array, pool, n, size)</literal> call to initialize an array object that has already been allocated. </para> <programlisting> ngx_array_t *a, b; /* create an array of strings with preallocated memory for 10 elements */ a = ngx_array_create(pool, 10, sizeof(ngx_str_t)); /* initialize string array for 10 elements */ ngx_array_init(&b, pool, 10, sizeof(ngx_str_t)); </programlisting> <para> Use the following functions to add elements to an array: </para> <para> <list type="bullet"> <listitem> <literal>ngx_array_push(a)</literal> adds one tail element and returns pointer to it </listitem> <listitem> <literal>ngx_array_push_n(a, n)</literal> adds <literal>n</literal> tail elements and returns pointer to the first one </listitem> </list> </para> <para> If the currently allocated amount of memory is not large enough to accommodate the new elements, a new block of memory is allocated and the existing elements are copied to it. The new memory block is normally twice as large as the existing one. </para> <programlisting> s = ngx_array_push(a); ss = ngx_array_push_n(&b, 3); </programlisting> </section> <section name="List" id="list"> <para> In nginx a list is a sequence of arrays, optimized for inserting a potentially large number of items. The <literal>ngx_list_t</literal> list type is defined as follows: </para> <programlisting> typedef struct { ngx_list_part_t *last; ngx_list_part_t part; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_list_t; </programlisting> <para> The actual items are stored in list parts, which are defined as follows: </para> <programlisting> typedef struct ngx_list_part_s ngx_list_part_t; struct ngx_list_part_s { void *elts; ngx_uint_t nelts; ngx_list_part_t *next; }; </programlisting> <para> Before use, a list must be initialized by calling <literal>ngx_list_init(list, pool, n, size)</literal> or created by calling <literal>ngx_list_create(pool, n, size)</literal>. Both functions take as arguments the size of a single item and a number of items per list part. To add an item to a list, use the <literal>ngx_list_push(list)</literal> function. To iterate over the items, directly access the list fields as shown in the example: </para> <programlisting> ngx_str_t *v; ngx_uint_t i; ngx_list_t *list; ngx_list_part_t *part; list = ngx_list_create(pool, 100, sizeof(ngx_str_t)); if (list == NULL) { /* error */ } /* add items to the list */ v = ngx_list_push(list); if (v == NULL) { /* error */ } ngx_str_set(v, "foo"); v = ngx_list_push(list); if (v == NULL) { /* error */ } ngx_str_set(v, "bar"); /* iterate over the list */ part = &list->part; v = part->elts; for (i = 0; /* void */; i++) { if (i >= part->nelts) { if (part->next == NULL) { break; } part = part->next; v = part->elts; i = 0; } ngx_do_smth(&v[i]); } </programlisting> <para> Lists are primarily used for HTTP input and output headers. </para> <para> Lists do not support item removal. However, when needed, items can internally be marked as missing without actually being removed from the list. For example, to mark HTTP output headers (which are stored as <literal>ngx_table_elt_t</literal> objects) as missing, set the <literal>hash</literal> field in <literal>ngx_table_elt_t</literal> to zero. Items marked in this way are explicitly skipped when the headers are iterated over. </para> </section> <section name="Queue" id="queue"> <para> In nginx a queue is an intrusive doubly linked list, with each node defined as follows: </para> <programlisting> typedef struct ngx_queue_s ngx_queue_t; struct ngx_queue_s { ngx_queue_t *prev; ngx_queue_t *next; }; </programlisting> <para> The head queue node is not linked with any data. Use the <literal>ngx_queue_init(q)</literal> call to initialize the list head before use. Queues support the following operations: </para> <para> <list type="bullet"> <listitem> <literal>ngx_queue_insert_head(h, x)</literal>, <literal>ngx_queue_insert_tail(h, x)</literal> — Insert a new node </listitem> <listitem> <literal>ngx_queue_remove(x)</literal> — Remove a queue node </listitem> <listitem> <literal>ngx_queue_split(h, q, n)</literal> — Split a queue at a node, returning the queue tail in a separate queue </listitem> <listitem> <literal>ngx_queue_add(h, n)</literal> — Add a second queue to the first queue </listitem> <listitem> <literal>ngx_queue_head(h)</literal>, <literal>ngx_queue_last(h)</literal> — Get first or last queue node </listitem> <listitem> <literal>ngx_queue_sentinel(h)</literal> - Get a queue sentinel object to end iteration at </listitem> <listitem> <literal>ngx_queue_data(q, type, link)</literal> — Get a reference to the beginning of a queue node data structure, considering the queue field offset in it </listitem> </list> </para> <para> An example: </para> <programlisting> typedef struct { ngx_str_t value; ngx_queue_t queue; } ngx_foo_t; ngx_foo_t *f; ngx_queue_t values, *q; ngx_queue_init(&values); f = ngx_palloc(pool, sizeof(ngx_foo_t)); if (f == NULL) { /* error */ } ngx_str_set(&f->value, "foo"); ngx_queue_insert_tail(&values, &f->queue); /* insert more nodes here */ for (q = ngx_queue_head(&values); q != ngx_queue_sentinel(&values); q = ngx_queue_next(q)) { f = ngx_queue_data(q, ngx_foo_t, queue); ngx_do_smth(&f->value); } </programlisting> </section> <section name="Red-Black tree" id="red_black_tree"> <para> The <path>src/core/ngx_rbtree.h</path> header file provides access to the effective implementation of red-black trees. </para> <programlisting> typedef struct { ngx_rbtree_t rbtree; ngx_rbtree_node_t sentinel; /* custom per-tree data here */ } my_tree_t; typedef struct { ngx_rbtree_node_t rbnode; /* custom per-node data */ foo_t val; } my_node_t; </programlisting> <para> To deal with a tree as a whole, you need two nodes: root and sentinel. Typically, they are added to a custom structure, allowing you to organize your data into a tree in which the leaves contain a link to or embed your data. </para> <para> To initialize a tree: </para> <programlisting> my_tree_t root; ngx_rbtree_init(&root.rbtree, &root.sentinel, insert_value_function); </programlisting> <para> To traverse a tree and insert new values, use the "<literal>insert_value</literal>" functions. For example, the <literal>ngx_str_rbtree_insert_value</literal> function deals with the <literal>ngx_str_t</literal> type. Its arguments are pointers to a root node of an insertion, the newly created node to be added, and a tree sentinel. </para> <programlisting> void ngx_str_rbtree_insert_value(ngx_rbtree_node_t *temp, ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel) </programlisting> <para> The traversal is pretty straightforward and can be demonstrated with the following lookup function pattern: </para> <programlisting> my_node_t * my_rbtree_lookup(ngx_rbtree_t *rbtree, foo_t *val, uint32_t hash) { ngx_int_t rc; my_node_t *n; ngx_rbtree_node_t *node, *sentinel; node = rbtree->root; sentinel = rbtree->sentinel; while (node != sentinel) { n = (my_node_t *) node; if (hash != node->key) { node = (hash < node->key) ? node->left : node->right; continue; } rc = compare(val, node->val); if (rc < 0) { node = node->left; continue; } if (rc > 0) { node = node->right; continue; } return n; } return NULL; } </programlisting> <para> The <literal>compare()</literal> function is a classic comparator function that returns a value less than, equal to, or greater than zero. To speed up lookups and avoid comparing user objects that can be big, an integer hash field is used. </para> <para> To add a node to a tree, allocate a new node, initialize it and call <literal>ngx_rbtree_insert()</literal>: </para> <programlisting> my_node_t *my_node; ngx_rbtree_node_t *node; my_node = ngx_palloc(...); init_custom_data(&my_node->val); node = &my_node->rbnode; node->key = create_key(my_node->val); ngx_rbtree_insert(&root->rbtree, node); </programlisting> <para> To remove a node, call the <literal>ngx_rbtree_delete()</literal> function: </para> <programlisting> ngx_rbtree_delete(&root->rbtree, node); </programlisting> </section> <section name="Hash" id="hash"> <para> Hash table functions are declared in <path>src/core/ngx_hash.h</path>. Both exact and wildcard matching are supported. The latter requires extra setup and is described in a separate section below. </para> <para> Before initializing a hash, you need to know the number of elements it will hold so that nginx can build it optimally. Two parameters that need to be configured are <literal>max_size</literal> and <literal>bucket_size</literal>, as detailed in a separate <link doc="../hash.xml">document</link>. They are usually configurable by the user. Hash initialization settings are stored with the <literal>ngx_hash_init_t</literal> type, and the hash itself is <literal>ngx_hash_t</literal>: <programlisting> ngx_hash_t foo_hash; ngx_hash_init_t hash; hash.hash = &foo_hash; hash.key = ngx_hash_key; hash.max_size = 512; hash.bucket_size = ngx_align(64, ngx_cacheline_size); hash.name = "foo_hash"; hash.pool = cf->pool; hash.temp_pool = cf->temp_pool; </programlisting> The <literal>key</literal> is a pointer to a function that creates the hash integer key from a string. There are two generic key-creation functions: <literal>ngx_hash_key(data, len)</literal> and <literal>ngx_hash_key_lc(data, len)</literal>. The latter converts a string to all lowercase characters, so the passed string must be writable. If that is not true, pass the <literal>NGX_HASH_READONLY_KEY</literal> flag to the function, initializing the key array (see below). </para> <para> The hash keys are stored in <literal>ngx_hash_keys_arrays_t</literal> and are initialized with <literal>ngx_hash_keys_array_init(arr, type)</literal>: The second parameter (<literal>type</literal>) controls the amount of resources preallocated for the hash and can be either <literal>NGX_HASH_SMALL</literal> or <literal>NGX_HASH_LARGE</literal>. The latter is appropriate if you expect the hash to contain thousands of elements. <programlisting> ngx_hash_keys_arrays_t foo_keys; foo_keys.pool = cf->pool; foo_keys.temp_pool = cf->temp_pool; ngx_hash_keys_array_init(&foo_keys, NGX_HASH_SMALL); </programlisting> </para> <para> To insert keys into a hash keys array, use the <literal>ngx_hash_add_key(keys_array, key, value, flags)</literal> function: <programlisting> ngx_str_t k1 = ngx_string("key1"); ngx_str_t k2 = ngx_string("key2"); ngx_hash_add_key(&foo_keys, &k1, &my_data_ptr_1, NGX_HASH_READONLY_KEY); ngx_hash_add_key(&foo_keys, &k2, &my_data_ptr_2, NGX_HASH_READONLY_KEY); </programlisting> </para> <para> To build the hash table, call the <literal>ngx_hash_init(hinit, key_names, nelts)</literal> function: <programlisting> ngx_hash_init(&hash, foo_keys.keys.elts, foo_keys.keys.nelts); </programlisting> The function fails if <literal>max_size</literal> or <literal>bucket_size</literal> parameters are not big enough. </para> <para> When the hash is built, use the <literal>ngx_hash_find(hash, key, name, len)</literal> function to look up elements: <programlisting> my_data_t *data; ngx_uint_t key; key = ngx_hash_key(k1.data, k1.len); data = ngx_hash_find(&foo_hash, key, k1.data, k1.len); if (data == NULL) { /* key not found */ } </programlisting> </para> <section name="Wildcard matching" id="wildcard_matching"> <para> To create a hash that works with wildcards, use the <literal>ngx_hash_combined_t</literal> type. It includes the hash type described above and has two additional keys arrays: <literal>dns_wc_head</literal> and <literal>dns_wc_tail</literal>. The initialization of basic properties is similar to a regular hash: <programlisting> ngx_hash_init_t hash ngx_hash_combined_t foo_hash; hash.hash = &foo_hash.hash; hash.key = ...; </programlisting> </para> <para> It is possible to add wildcard keys using the <literal>NGX_HASH_WILDCARD_KEY</literal> flag: <programlisting> /* k1 = ".example.org"; */ /* k2 = "foo.*"; */ ngx_hash_add_key(&foo_keys, &k1, &data1, NGX_HASH_WILDCARD_KEY); ngx_hash_add_key(&foo_keys, &k2, &data2, NGX_HASH_WILDCARD_KEY); </programlisting> The function recognizes wildcards and adds keys into the corresponding arrays. Please refer to the <link doc="../http/ngx_http_map_module.xml" id="map"/> module documentation for the description of the wildcard syntax and the matching algorithm. </para> <para> Depending on the contents of added keys, you may need to initialize up to three key arrays: one for exact matching (described above), and two more to enable matching starting from the head or tail of a string: <programlisting> if (foo_keys.dns_wc_head.nelts) { ngx_qsort(foo_keys.dns_wc_head.elts, (size_t) foo_keys.dns_wc_head.nelts, sizeof(ngx_hash_key_t), cmp_dns_wildcards); hash.hash = NULL; hash.temp_pool = pool; if (ngx_hash_wildcard_init(&hash, foo_keys.dns_wc_head.elts, foo_keys.dns_wc_head.nelts) != NGX_OK) { return NGX_ERROR; } foo_hash.wc_head = (ngx_hash_wildcard_t *) hash.hash; } </programlisting> The keys array needs to be sorted, and initialization results must be added to the combined hash. The initialization of <literal>dns_wc_tail</literal> array is done similarly. </para> <para> The lookup in a combined hash is handled by the <literal>ngx_hash_find_combined(chash, key, name, len)</literal>: <programlisting> /* key = "bar.example.org"; — will match ".example.org" */ /* key = "foo.example.com"; — will match "foo.*" */ hkey = ngx_hash_key(key.data, key.len); res = ngx_hash_find_combined(&foo_hash, hkey, key.data, key.len); </programlisting> </para> </section> </section> </section> <section name="Memory management" id="memory_management"> <section name="Heap" id="heap"> <para> To allocate memory from system heap, use the following functions: </para> <para> <list type="bullet"> <listitem> <literal>ngx_alloc(size, log)</literal> — Allocate memory from system heap. This is a wrapper around <literal>malloc()</literal> with logging support. Allocation error and debugging information is logged to <literal>log</literal>. </listitem> <listitem> <literal>ngx_calloc(size, log)</literal> — Allocate memory from system heap like <literal>ngx_alloc()</literal>, but fill memory with zeros after allocation. </listitem> <listitem> <literal>ngx_memalign(alignment, size, log)</literal> — Allocate aligned memory from system heap. This is a wrapper around <literal>posix_memalign()</literal> on those platforms that provide that function. Otherwise implementation falls back to <literal>ngx_alloc()</literal> which provides maximum alignment. </listitem> <listitem> <literal>ngx_free(p)</literal> — Free allocated memory. This is a wrapper around <literal>free()</literal> </listitem> </list> </para> </section> <section name="Pool" id="pool"> <para> Most nginx allocations are done in pools. Memory allocated in an nginx pool is freed automatically when the pool is destroyed. This provides good allocation performance and makes memory control easy. </para> <para> A pool internally allocates objects in continuous blocks of memory. Once a block is full, a new one is allocated and added to the pool memory block list. When the requested allocation is too large to fit into a block, the request is forwarded to the system allocator and the returned pointer is stored in the pool for further deallocation. </para> <para> The type for nginx pools is <literal>ngx_pool_t</literal>. The following operations are supported: </para> <para> <list type="bullet"> <listitem> <literal>ngx_create_pool(size, log)</literal> — Create a pool with specified block size. The pool object returned is allocated in the pool as well. The <literal>size</literal> should be at least <literal>NGX_MIN_POOL_SIZE</literal> and a multiple of <literal>NGX_POOL_ALIGNMENT</literal>. </listitem> <listitem> <literal>ngx_destroy_pool(pool)</literal> — Free all pool memory, including the pool object itself. </listitem> <listitem> <literal>ngx_palloc(pool, size)</literal> — Allocate aligned memory from the specified pool. </listitem> <listitem> <literal>ngx_pcalloc(pool, size)</literal> — Allocate aligned memory from the specified pool and fill it with zeroes. </listitem> <listitem> <literal>ngx_pnalloc(pool, size)</literal> — Allocate unaligned memory from the specified pool. Mostly used for allocating strings. </listitem> <listitem> <literal>ngx_pfree(pool, p)</literal> — Free memory that was previously allocated in the specified pool. Only allocations that result from requests forwarded to the system allocator can be freed. </listitem> </list> </para> <programlisting> u_char *p; ngx_str_t *s; ngx_pool_t *pool; pool = ngx_create_pool(1024, log); if (pool == NULL) { /* error */ } s = ngx_palloc(pool, sizeof(ngx_str_t)); if (s == NULL) { /* error */ } ngx_str_set(s, "foo"); p = ngx_pnalloc(pool, 3); if (p == NULL) { /* error */ } ngx_memcpy(p, "foo", 3); </programlisting> <para> Chain links (<literal>ngx_chain_t</literal>) are actively used in nginx, so the nginx pool implementation provides a way to reuse them. The <literal>chain</literal> field of <literal>ngx_pool_t</literal> keeps a list of previously allocated links ready for reuse. For efficient allocation of a chain link in a pool, use the <literal>ngx_alloc_chain_link(pool)</literal> function. This function looks up a free chain link in the pool list and allocates a new chain link if the pool list is empty. To free a link, call the <literal>ngx_free_chain(pool, cl)</literal> function. </para> <para> Cleanup handlers can be registered in a pool. A cleanup handler is a callback with an argument which is called when pool is destroyed. A pool is usually tied to a specific nginx object (like an HTTP request) and is destroyed when the object reaches the end of its lifetime. Registering a pool cleanup is a convenient way to release resources, close file descriptors or make final adjustments to the shared data associated with the main object. </para> <para> To register a pool cleanup, call <literal>ngx_pool_cleanup_add(pool, size)</literal>, which returns a <literal>ngx_pool_cleanup_t</literal> pointer to be filled in by the caller. Use the <literal>size</literal> argument to allocate context for the cleanup handler. </para> <programlisting> ngx_pool_cleanup_t *cln; cln = ngx_pool_cleanup_add(pool, 0); if (cln == NULL) { /* error */ } cln->handler = ngx_my_cleanup; cln->data = "foo"; ... static void ngx_my_cleanup(void *data) { u_char *msg = data; ngx_do_smth(msg); } </programlisting> </section> <section name="Shared memory" id="shared_memory"> <para> Shared memory is used by nginx to share common data between processes. The <literal>ngx_shared_memory_add(cf, name, size, tag)</literal> function adds a new shared memory entry <literal>ngx_shm_zone_t</literal> to a cycle. The function receives the <literal>name</literal> and <literal>size</literal> of the zone. Each shared zone must have a unique name. If a shared zone entry with the provided <literal>name</literal> and <literal>tag</literal> already exists, the existing zone entry is reused. The function fails with an error if an existing entry with the same name has a different tag. Usually, the address of the module structure is passed as <literal>tag</literal>, making it possible to reuse shared zones by name within one nginx module. </para> <para> The shared memory entry structure <literal>ngx_shm_zone_t</literal> has the following fields: </para> <para> <list type="bullet"> <listitem> <literal>init</literal> — Initialization callback, called after the shared zone is mapped to actual memory </listitem> <listitem> <literal>data</literal> — Data context, used to pass arbitrary data to the <literal>init</literal> callback </listitem> <listitem> <literal>noreuse</literal> — Flag that disables reuse of a shared zone from the old cycle </listitem> <listitem> <literal>tag</literal> — Shared zone tag </listitem> <listitem> <literal>shm</literal> — Platform-specific object of type <literal>ngx_shm_t</literal>, having at least the following fields: <list type="bullet"> <listitem> <literal>addr</literal> — Mapped shared memory address, initially NULL </listitem> <listitem> <literal>size</literal> — Shared memory size </listitem> <listitem> <literal>name</literal> — Shared memory name </listitem> <listitem> <literal>log</literal> — Shared memory log </listitem> <listitem> <literal>exists</literal> — Flag that indicates shared memory was inherited from the master process (Windows-specific) </listitem> </list> </listitem> </list> </para> <para> Shared zone entries are mapped to actual memory in <literal>ngx_init_cycle()</literal> after the configuration is parsed. On POSIX systems, the <literal>mmap()</literal> syscall is used to create the shared anonymous mapping. On Windows, the <literal>CreateFileMapping()</literal>/ <literal>MapViewOfFileEx()</literal> pair is used. </para> <para> For allocating in shared memory, nginx provides the slab pool <literal>ngx_slab_pool_t</literal> type. A slab pool for allocating memory is automatically created in each nginx shared zone. The pool is located in the beginning of the shared zone and can be accessed by the expression <literal>(ngx_slab_pool_t *) shm_zone->shm.addr</literal>. To allocate memory in a shared zone, call either <literal>ngx_slab_alloc(pool, size)</literal> or <literal>ngx_slab_calloc(pool, size)</literal>. To free memory, call <literal>ngx_slab_free(pool, p)</literal>. </para> <para> Slab pool divides all shared zone into pages. Each page is used for allocating objects of the same size. The specified size must be a power of 2, and greater than the minimum size of 8 bytes. Nonconforming values are rounded up. A bitmask for each page tracks which blocks are in use and which are free for allocation. For sizes greater than a half page (which is usually 2048 bytes), allocation is done an entire page at a time </para> <para> To protect data in shared memory from concurrent access, use the mutex available in the <literal>mutex</literal> field of <literal>ngx_slab_pool_t</literal>. A mutex is most commonly used by the slab pool while allocating and freeing memory, but it can be used to protect any other user data structures allocated in the shared zone. To lock or unlock a mutex, call <literal>ngx_shmtx_lock(&shpool->mutex)</literal> or <literal>ngx_shmtx_unlock(&shpool->mutex)</literal> respectively. </para> <programlisting> ngx_str_t name; ngx_foo_ctx_t *ctx; ngx_shm_zone_t *shm_zone; ngx_str_set(&name, "foo"); /* allocate shared zone context */ ctx = ngx_pcalloc(cf->pool, sizeof(ngx_foo_ctx_t)); if (ctx == NULL) { /* error */ } /* add an entry for 64k shared zone */ shm_zone = ngx_shared_memory_add(cf, &name, 65536, &ngx_foo_module); if (shm_zone == NULL) { /* error */ } /* register init callback and context */ shm_zone->init = ngx_foo_init_zone; shm_zone->data = ctx; ... static ngx_int_t ngx_foo_init_zone(ngx_shm_zone_t *shm_zone, void *data) { ngx_foo_ctx_t *octx = data; size_t len; ngx_foo_ctx_t *ctx; ngx_slab_pool_t *shpool; value = shm_zone->data; if (octx) { /* reusing a shared zone from old cycle */ ctx->value = octx->value; return NGX_OK; } shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; if (shm_zone->shm.exists) { /* initialize shared zone context in Windows nginx worker */ ctx->value = shpool->data; return NGX_OK; } /* initialize shared zone */ ctx->value = ngx_slab_alloc(shpool, sizeof(ngx_uint_t)); if (ctx->value == NULL) { return NGX_ERROR; } shpool->data = ctx->value; return NGX_OK; } </programlisting> </section> </section> <section name="Logging" id="logging"> <para> For logging nginx uses <literal>ngx_log_t</literal> objects. The nginx logger supports several types of output: <list type="bullet"> <listitem> stderr — Logging to standard error (stderr) </listitem> <listitem> file — Logging to a file </listitem> <listitem> syslog — Logging to syslog </listitem> <listitem> memory — Logging to internal memory storage for development purposes; the memory can be accessed later with a debugger </listitem> </list> </para> <para> A logger instance can be a chain of loggers, linked to each other with the <literal>next</literal> field. In this case, each message is written to all loggers in the chain. </para> <para> For each logger, a severity level controls which messages are written to the log (only events assigned that level or higher are logged). The following severity levels are supported: </para> <para> <list type="bullet"> <listitem> <literal>NGX_LOG_EMERG</literal> </listitem> <listitem> <literal>NGX_LOG_ALERT</literal> </listitem> <listitem> <literal>NGX_LOG_CRIT</literal> </listitem> <listitem> <literal>NGX_LOG_ERR</literal> </listitem> <listitem> <literal>NGX_LOG_WARN</literal> </listitem> <listitem> <literal>NGX_LOG_NOTICE</literal> </listitem> <listitem> <literal>NGX_LOG_INFO</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG</literal> </listitem> </list> </para> <para> For debug logging, the debug mask is checked as well. The debug masks are: </para> <para> <list type="bullet"> <listitem> <literal>NGX_LOG_DEBUG_CORE</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG_ALLOC</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG_MUTEX</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG_EVENT</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG_HTTP</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG_MAIL</literal> </listitem> <listitem> <literal>NGX_LOG_DEBUG_STREAM</literal> </listitem> </list> </para> <para> Normally, loggers are created by existing nginx code from <literal>error_log</literal> directives and are available at nearly every stage of processing in cycle, configuration, client connection and other objects. </para> <para> Nginx provides the following logging macros: </para> <para> <list type="bullet"> <listitem> <literal>ngx_log_error(level, log, err, fmt, ...)</literal> — Error logging </listitem> <listitem> <literal>ngx_log_debug0(level, log, err, fmt)</literal>, <literal>ngx_log_debug1(level, log, err, fmt, arg1)</literal> etc — Debug logging with up to eight supported formatting arguments </listitem> </list> </para> <para> A log message is formatted in a buffer of size <literal>NGX_MAX_ERROR_STR</literal> (currently, 2048 bytes) on stack. The message is prepended with the severity level, process ID (PID), connection ID (stored in <literal>log->connection</literal>), and the system error text. For non-debug messages <literal>log->handler</literal> is called as well to prepend more specific information to the log message. HTTP module sets <literal>ngx_http_log_error()</literal> function as log handler to log client and server addresses, current action (stored in <literal>log->action</literal>), client request line, server name etc. </para> <programlisting> /* specify what is currently done */ log->action = "sending mp4 to client"; /* error and debug log */ ngx_log_error(NGX_LOG_INFO, c->log, 0, "client prematurely closed connection"); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "mp4 start:%ui, length:%ui", mp4->start, mp4->length); </programlisting> <para> The example above results in log entries like these: </para> <programlisting> 2016/09/16 22:08:52 [info] 17445#0: *1 client prematurely closed connection while sending mp4 to client, client: 127.0.0.1, server: , request: "GET /file.mp4 HTTP/1.1" 2016/09/16 23:28:33 [debug] 22140#0: *1 mp4 start:0, length:10000 </programlisting> </section> <section name="Cycle" id="cycle"> <para> A cycle object stores the nginx runtime context created from a specific configuration. Its type is <literal>ngx_cycle_t</literal>. The current cycle is referenced by the <literal>ngx_cycle</literal> global variable and inherited by nginx workers as they start. Each time the nginx configuration is reloaded, a new cycle is created from the new nginx configuration; the old cycle is usually deleted after the new one is successfully created. </para> <para> A cycle is created by the <literal>ngx_init_cycle()</literal> function, which takes the previous cycle as its argument. The function locates the previous cycle's configuration file and inherits as many resources as possible from the previous cycle. A placeholder cycle called "init cycle" is created as nginx start, then is replaced by an actual cycle built from configuration. </para> <para> Members of the cycle include: </para> <para> <list type="bullet"> <listitem> <literal>pool</literal> — Cycle pool. Created for each new cycle. </listitem> <listitem> <literal>log</literal> — Cycle log. Initially inherited from the old cycle, it is set to point to <literal>new_log</literal> after the configuration is read. </listitem> <listitem> <literal>new_log</literal> — Cycle log, created by the configuration. It's affected by the root-scope <literal>error_log</literal> directive. </listitem> <listitem> <literal>connections</literal>, <literal>connection_n</literal> — Array of connections of type <literal>ngx_connection_t</literal>, created by the event module while initializing each nginx worker. The <literal>worker_connections</literal> directive in the nginx configuration sets the number of connections <literal>connection_n</literal>. </listitem> <listitem> <literal>free_connections</literal>, <literal>free_connection_n</literal> — List and number of currently available connections. If no connections are available, an nginx worker refuses to accept new clients or connect to upstream servers. </listitem> <listitem> <literal>files</literal>, <literal>files_n</literal> — Array for mapping file descriptors to nginx connections. This mapping is used by the event modules, having the <literal>NGX_USE_FD_EVENT</literal> flag (currently, it's <literal>poll</literal> and <literal>devpoll</literal>). </listitem> <listitem> <literal>conf_ctx</literal> — Array of core module configurations. The configurations are created and filled during reading of nginx configuration files. </listitem> <listitem> <literal>modules</literal>, <literal>modules_n</literal> — Array of modules of type <literal>ngx_module_t</literal>, both static and dynamic, loaded by the current configuration. </listitem> <listitem> <literal>listening</literal> — Array of listening objects of type <literal>ngx_listening_t</literal>. Listening objects are normally added by the <literal>listen</literal> directive of different modules which call the <literal>ngx_create_listening()</literal> function. Listen sockets are created based on the listening objects. </listitem> <listitem> <literal>paths</literal> — Array of paths of type <literal>ngx_path_t</literal>. Paths are added by calling the function <literal>ngx_add_path()</literal> from modules which are going to operate on certain directories. These directories are created by nginx after reading configuration, if missing. Moreover, two handlers can be added for each path: <list type="bullet"> <listitem> path loader — Executes only once in 60 seconds after starting or reloading nginx. Normally, the loader reads the directory and stores data in nginx shared memory. The handler is called from the dedicated nginx process “nginx cache loader”. </listitem> <listitem> path manager — Executes periodically. Normally, the manager removes old files from the directory and updates nginx memory to reflect the changes. The handler is called from the dedicated “nginx cache manager” process. </listitem> </list> </listitem> <listitem> <literal>open_files</literal> — List of open file objects of type <literal>ngx_open_file_t</literal>, which are created by calling the function <literal>ngx_conf_open_file()</literal>. Currently, nginx uses this kind of open files for logging. After reading the configuration, nginx opens all files in the <literal>open_files</literal> list and stores each file descriptor in the object's <literal>fd</literal> field. The files are opened in append mode and are created if missing. The files in the list are reopened by nginx workers upon receiving the reopen signal (most often <literal>USR1</literal>). In this case the descriptor in the <literal>fd</literal> field is changed to a new value. </listitem> <listitem> <literal>shared_memory</literal> — List of shared memory zones, each added by calling the <literal>ngx_shared_memory_add()</literal> function. Shared zones are mapped to the same address range in all nginx processes and are used to share common data, for example the HTTP cache in-memory tree. </listitem> </list> </para> </section> <section name="Buffer" id="buffer"> <para> For input/output operations, nginx provides the buffer type <literal>ngx_buf_t</literal>. Normally, it's used to hold data to be written to a destination or read from a source. A buffer can reference data in memory or in a file and it's technically possible for a buffer to reference both at the same time. Memory for the buffer is allocated separately and is not related to the buffer structure <literal>ngx_buf_t</literal>. </para> <para> The <literal>ngx_buf_t</literal> structure has the following fields: </para> <para> <list type="bullet"> <listitem> <literal>start</literal>, <literal>end</literal> — The boundaries of the memory block allocated for the buffer. </listitem> <listitem> <literal>pos</literal>, <literal>last</literal> — The boundaries of the memory buffer; normally a subrange of <literal>start</literal> .. <literal>end</literal>. </listitem> <listitem> <literal>file_pos</literal>, <literal>file_last</literal> — The boundaries of a file buffer, expressed as offsets from the beginning of the file. </listitem> <listitem> <literal>tag</literal> — Unique value used to distinguish buffers; created by different nginx modules, usually for the purpose of buffer reuse. </listitem> <listitem> <literal>file</literal> — File object. </listitem> <listitem> <literal>temporary</literal> — Flag indicating that the buffer references writable memory. </listitem> <listitem> <literal>memory</literal> — Flag indicating that the buffer references read-only memory. </listitem> <listitem> <literal>in_file</literal> — Flag indicating that the buffer references data in a file. </listitem> <listitem> <literal>flush</literal> — Flag indicating that all data prior to the buffer need to be flushed. </listitem> <listitem> <literal>recycled</literal> — Flag indicating that the buffer can be reused and needs to be consumed as soon as possible. </listitem> <listitem> <literal>sync</literal> — Flag indicating that the buffer carries no data or special signal like <literal>flush</literal> or <literal>last_buf</literal>. By default nginx considers such buffers an error condition, but this flag tells nginx to skip the error check. </listitem> <listitem> <literal>last_buf</literal> — Flag indicating that the buffer is the last in output. </listitem> <listitem> <literal>last_in_chain</literal> — Flag indicating that there are no more data buffers in a request or subrequest. </listitem> <listitem> <literal>shadow</literal> — Reference to another ("shadow") buffer related to the current buffer, usually in the sense that the buffer uses data from the shadow. When the buffer is consumed, the shadow buffer is normally also marked as consumed. </listitem> <listitem> <literal>last_shadow</literal> — Flag indicating that the buffer is the last one that references a particular shadow buffer. </listitem> <listitem> <literal>temp_file</literal> — Flag indicating that the buffer is in a temporary file. </listitem> </list> </para> <para> For input and output operations buffers are linked in chains. A chain is a sequence of chain links of type <literal>ngx_chain_t</literal>, defined as follows: </para> <programlisting> typedef struct ngx_chain_s ngx_chain_t; struct ngx_chain_s { ngx_buf_t *buf; ngx_chain_t *next; }; </programlisting> <para> Each chain link keeps a reference to its buffer and a reference to the next chain link. </para> <para> An example of using buffers and chains: </para> <programlisting> ngx_chain_t * ngx_get_my_chain(ngx_pool_t *pool) { ngx_buf_t *b; ngx_chain_t *out, *cl, **ll; /* first buf */ cl = ngx_alloc_chain_link(pool); if (cl == NULL) { /* error */ } b = ngx_calloc_buf(pool); if (b == NULL) { /* error */ } b->start = (u_char *) "foo"; b->pos = b->start; b->end = b->start + 3; b->last = b->end; b->memory = 1; /* read-only memory */ cl->buf = b; out = cl; ll = &cl->next; /* second buf */ cl = ngx_alloc_chain_link(pool); if (cl == NULL) { /* error */ } b = ngx_create_temp_buf(pool, 3); if (b == NULL) { /* error */ } b->last = ngx_cpymem(b->last, "foo", 3); cl->buf = b; cl->next = NULL; *ll = cl; return out; } </programlisting> </section> <section name="Networking" id="networking"> <!-- <section name="Network data types" id="network_data_types"> <para> TBD: ngx_addr_t, ngx_url_t, ngx_socket_t, ngx_sockaddr_t, parse url, parse address... </para> </section> --> <section name="Connection" id="connection"> <para> The connection type <literal>ngx_connection_t</literal> is a wrapper around a socket descriptor. It includes the following fields: </para> <para> <list type="bullet"> <listitem> <literal>fd</literal> — Socket descriptor </listitem> <listitem> <literal>data</literal> — Arbitrary connection context. Normally, it is a pointer to a higher-level object built on top of the connection, such as an HTTP request or a Stream session. </listitem> <listitem> <literal>read</literal>, <literal>write</literal> — Read and write events for the connection. </listitem> <listitem> <literal>recv</literal>, <literal>send</literal>, <literal>recv_chain</literal>, <literal>send_chain</literal> — I/O operations for the connection. </listitem> <listitem> <literal>pool</literal> — Connection pool. </listitem> <listitem> <literal>log</literal> — Connection log. </listitem> <listitem> <literal>sockaddr</literal>, <literal>socklen</literal>, <literal>addr_text</literal> — Remote socket address in binary and text forms. </listitem> <listitem> <literal>local_sockaddr</literal>, <literal>local_socklen</literal> — Local socket address in binary form. Initially, these fields are empty. Use the <literal>ngx_connection_local_sockaddr()</literal> function to get the local socket address. </listitem> <listitem> <literal>proxy_protocol_addr</literal>, <literal>proxy_protocol_port</literal> - PROXY protocol client address and port, if the PROXY protocol is enabled for the connection. </listitem> <listitem> <literal>ssl</literal> — SSL context for the connection. </listitem> <listitem> <literal>reusable</literal> — Flag indicating the connection is in a state that makes it eligible for reuse. </listitem> <listitem> <literal>close</literal> — Flag indicating that the connection is being reused and needs to be closed. </listitem> </list> </para> <para> An nginx connection can transparently encapsulate the SSL layer. In this case the connection's <literal>ssl</literal> field holds a pointer to an <literal>ngx_ssl_connection_t</literal> structure, keeping all SSL-related data for the connection, including <literal>SSL_CTX</literal> and <literal>SSL</literal>. The <literal>recv</literal>, <literal>send</literal>, <literal>recv_chain</literal>, and <literal>send_chain</literal> handlers are set to SSL-enabled functions as well. </para> <para> The <literal>worker_connections</literal> directive in the nginx configuration limits the number of connections per nginx worker. All connection structures are precreated when a worker starts and stored in the <literal>connections</literal> field of the cycle object. To retrieve a connection structure, use the <literal>ngx_get_connection(s, log)</literal> function. It takes as its <literal>s</literal> argument a socket descriptor, which needs to be wrapped in a connection structure. </para> <para> Because the number of connections per worker is limited, nginx provides a way to grab connections that are currently in use. To enable or disable reuse of a connection, call the <literal>ngx_reusable_connection(c, reusable)</literal> function. Calling <literal>ngx_reusable_connection(c, 1)</literal> sets the <literal>reuse</literal> flag in the connection structure and inserts the connection into the <literal>reusable_connections_queue</literal> of the cycle. Whenever <literal>ngx_get_connection()</literal> finds out there are no available connections in the cycle's <literal>free_connections</literal> list, it calls <literal>ngx_drain_connections()</literal> to release a specific number of reusable connections. For each such connection, the <literal>close</literal> flag is set and its read handler is called which is supposed to free the connection by calling <literal>ngx_close_connection(c)</literal> and make it available for reuse. To exit the state when a connection can be reused <literal>ngx_reusable_connection(c, 0)</literal> is called. HTTP client connections are an example of reusable connections in nginx; they are marked as reusable until the first request byte is received from the client. </para> </section> </section> <section name="Events" id="events"> <section name="Event" id="event"> <para> Event object <literal>ngx_event_t</literal> in nginx provides a mechanism for notification that a specific event has occurred. </para> <para> Fields in <literal>ngx_event_t</literal> include: </para> <para> <list type="bullet"> <listitem> <literal>data</literal> — Arbitrary event context used in event handlers, usually as pointer to a connection related to the event. </listitem> <listitem> <literal>handler</literal> — Callback function to be invoked when the event happens. </listitem> <listitem> <literal>write</literal> — Flag indicating a write event. Absence of the flag indicates a read event. </listitem> <listitem> <literal>active</literal> — Flag indicating that the event is registered for receiving I/O notifications, normally from notification mechanisms like <literal>epoll</literal>, <literal>kqueue</literal>, <literal>poll</literal>. </listitem> <listitem> <literal>ready</literal> — Flag indicating that the event has received an I/O notification. </listitem> <listitem> <literal>delayed</literal> — Flag indicating that I/O is delayed due to rate limiting. </listitem> <listitem> <literal>timer</literal> — Red-black tree node for inserting the event into the timer tree. </listitem> <listitem> <literal>timer_set</literal> — Flag indicating that the event timer is set and not yet expired. </listitem> <listitem> <literal>timedout</literal> — Flag indicating that the event timer has expired. </listitem> <listitem> <literal>eof</literal> — Flag indicating that EOF occurred while reading data. </listitem> <listitem> <literal>pending_eof</literal> — Flag indicating that EOF is pending on the socket, even though there may be some data available before it. The flag is delivered via the <literal>EPOLLRDHUP</literal> <literal>epoll</literal> event or <literal>EV_EOF</literal> <literal>kqueue</literal> flag. </listitem> <listitem> <literal>error</literal> — Flag indicating that an error occurred during reading (for a read event) or writing (for a write event). </listitem> <listitem> <literal>cancelable</literal> — Timer event flag indicating that the event should be ignored while shutting down the worker. Graceful worker shutdown is delayed until there are no non-cancelable timer events scheduled. </listitem> <listitem> <literal>posted</literal> — Flag indicating that the event is posted to a queue. </listitem> <listitem> <literal>queue</literal> — Queue node for posting the event to a queue. </listitem> </list> </para> </section> <section name="I/O events" id="i_o_events"> <para> Each connection obtained by calling the <literal>ngx_get_connection()</literal> function has two attached events, <literal>c->read</literal> and <literal>c->write</literal>, which are used for receiving notification that the socket is ready for reading or writing. All such events operate in Edge-Triggered mode, meaning that they only trigger notifications when the state of the socket changes. For example, doing a partial read on a socket does not make nginx deliver a repeated read notification until more data arrives on the socket. Even when the underlying I/O notification mechanism is essentially Level-Triggered (<literal>poll</literal>, <literal>select</literal> etc), nginx converts the notifications to Edge-Triggered. To make nginx event notifications consistent across all notifications systems on different platforms, the functions <literal>ngx_handle_read_event(rev, flags)</literal> and <literal>ngx_handle_write_event(wev, lowat)</literal> must be called after handling an I/O socket notification or calling any I/O functions on that socket. Normally, the functions are called once at the end of each read or write event handler. </para> </section> <section name="Timer events" id="timer_events"> <para> An event can be set to send a notification when a timeout expires. The timer used by events counts milliseconds since some unspecified point in the past truncated to <literal>ngx_msec_t</literal> type. Its current value can be obtained from the <literal>ngx_current_msec</literal> variable. </para> <para> The function <literal>ngx_add_timer(ev, timer)</literal> sets a timeout for an event, <literal>ngx_del_timer(ev)</literal> deletes a previously set timeout. The global timeout red-black tree <literal>ngx_event_timer_rbtree</literal> stores all timeouts currently set. The key in the tree is of type <literal>ngx_msec_t</literal> and is the time when the event occurs. The tree structure enables fast insertion and deletion operations, as well as access to the nearest timeouts, which nginx uses to find out how long to wait for I/O events and for expiring timeout events. </para> </section> <section name="Posted events" id="posted_events"> <para> An event can be posted which means that its handler will be called at some point later within the current event loop iteration. Posting events is a good practice for simplifying code and escaping stack overflows. Posted events are held in a post queue. The <literal>ngx_post_event(ev, q)</literal> macro posts the event <literal>ev</literal> to the post queue <literal>q</literal>. The <literal>ngx_delete_posted_event(ev)</literal> macro deletes the event <literal>ev</literal> from the queue it's currently posted in. Normally, events are posted to the <literal>ngx_posted_events</literal> queue, which is processed late in the event loop — after all I/O and timer events are already handled. The function <literal>ngx_event_process_posted()</literal> is called to process an event queue. It calls event handlers until the queue is not empty. This means that a posted event handler can post more events to be processed within the current event loop iteration. </para> <para> An example: </para> <programlisting> void ngx_my_connection_read(ngx_connection_t *c) { ngx_event_t *rev; rev = c->read; ngx_add_timer(rev, 1000); rev->handler = ngx_my_read_handler; ngx_my_read(rev); } void ngx_my_read_handler(ngx_event_t *rev) { ssize_t n; ngx_connection_t *c; u_char buf[256]; if (rev->timedout) { /* timeout expired */ } c = rev->data; while (rev->ready) { n = c->recv(c, buf, sizeof(buf)); if (n == NGX_AGAIN) { break; } if (n == NGX_ERROR) { /* error */ } /* process buf */ } if (ngx_handle_read_event(rev, 0) != NGX_OK) { /* error */ } } </programlisting> </section> <section name="Event loop" id="event_loop"> <para> Except for the nginx master process, all nginx processes do I/O and so have an event loop. (The nginx master process instead spends most of its time in the <literal>sigsuspend()</literal> call waiting for signals to arrive.) The nginx event loop is implemented in the <literal>ngx_process_events_and_timers()</literal> function, which is called repeatedly until the process exits. </para> <para> The event loop has the following stages: <list type="bullet"> <listitem> Find the timeout that is closest to expiring, by calling <literal>ngx_event_find_timer()</literal>. This function finds the leftmost node in the timer tree and returns the number of milliseconds until the node expires. </listitem> <listitem> Process I/O events by calling a handler, specific to the event notification mechanism, chosen by nginx configuration. This handler waits for at least one I/O event to happen, but only until the next timeout expires. When a read or write event occurs, the <literal>ready</literal> flag is set and the event's handler is called. For Linux, the <literal>ngx_epoll_process_events()</literal> handler is normally used, which calls <literal>epoll_wait()</literal> to wait for I/O events. </listitem> <listitem> Expire timers by calling <literal>ngx_event_expire_timers()</literal>. The timer tree is iterated from the leftmost element to the right until an unexpired timeout is found. For each expired node the <literal>timedout</literal> event flag is set, the <literal>timer_set</literal> flag is reset, and the event handler is called </listitem> <listitem> Process posted events by calling <literal>ngx_event_process_posted()</literal>. The function repeatedly removes the first element from the posted events queue and calls the element's handler, until the queue is empty. </listitem> </list> </para> <para> All nginx processes handle signals as well. Signal handlers only set global variables which are checked after the <literal>ngx_process_events_and_timers()</literal> call. </para> </section> </section> <section name="Processes" id="processes"> <para> There are several types of processes in nginx. The type of a process is kept in the <literal>ngx_process</literal> global variable, and is one of the following: </para> <list type="bullet"> <listitem> <para> <literal>NGX_PROCESS_MASTER</literal> — The master process, which reads the NGINX configuration, creates cycles, and starts and controls child processes. It does not perform any I/O and responds only to signals. Its cycle function is <literal>ngx_master_process_cycle()</literal>. </para> </listitem> <listitem> <para> <literal>NGX_PROCESS_WORKER</literal> — The worker process, which handles client connections. It is started by the master process and responds to its signals and channel commands as well. Its cycle function is <literal>ngx_worker_process_cycle()</literal>. There can be multiple worker processes, as configured by the <literal>worker_processes</literal> directive. </para> </listitem> <listitem> <para> <literal>NGX_PROCESS_SINGLE</literal> — The single process, which exists only in <literal>master_process off</literal> mode, and is the only process running in that mode. It creates cycles (like the master process does) and handles client connections (like the worker process does). Its cycle function is <literal>ngx_single_process_cycle()</literal>. </para> </listitem> <listitem> <para> <literal>NGX_PROCESS_HELPER</literal> — The helper process, of which currently there are two types: cache manager and cache loader. The cycle function for both is <literal>ngx_cache_manager_process_cycle()</literal>. </para> </listitem> </list> <para> The nginx processes handle the following signals: </para> <list type="bullet"> <listitem> <para> <literal>NGX_SHUTDOWN_SIGNAL</literal> (<literal>SIGQUIT</literal> on most systems) — Gracefully shutdown. Upon receiving this signal, the master process sends a shutdown signal to all child processes. When no child processes are left, the master destroys the cycle pool and exits. When a worker process receives this signal, it closes all listening sockets and waits until there are no non-cancelable events scheduled, then destroys the cycle pool and exits. When the cache manager or the cache loader process receives this signal, it exits immediately. The <literal>ngx_quit</literal> variable is set to <literal>1</literal> when a process receives this signal, and is immediately reset after being processed. The <literal>ngx_exiting</literal> variable is set to <literal>1</literal> while a worker process is in the shutdown state. </para> </listitem> <listitem> <para> <literal>NGX_TERMINATE_SIGNAL</literal> (<literal>SIGTERM</literal> on most systems) — Terminate. Upon receiving this signal, the master process sends a terminate signal to all child processes. If a child process does not exit within 1 second, the master process sends the <literal>SIGKILL</literal> signal to kill it. When no child processes are left, the master process destroys the cycle pool and exits. When a worker process, the cache manager process or the cache loader process receives this signal, it destroys the cycle pool and exits. The variable <literal>ngx_terminate</literal> is set to <literal>1</literal> when this signal is received. </para> </listitem> <listitem> <para> <literal>NGX_NOACCEPT_SIGNAL</literal> (<literal>SIGWINCH</literal> on most systems) - Shut down all worker and helper processes. Upon receiving this signal, the master process shuts down its child processes. If a previously started new nginx binary exits, the child processes of the old master are started again. When a worker process receives this signal, it shuts down in debug mode set by the <literal>debug_points</literal> directive. </para> </listitem> <listitem> <para> <literal>NGX_RECONFIGURE_SIGNAL</literal> (<literal>SIGHUP</literal> on most systems) - Reconfigure. Upon receiving this signal, the master process re-reads the configuration and creates a new cycle based on it. If the new cycle is created successfully, the old cycle is deleted and new child processes are started. Meanwhile, the old child processes receive the <literal>NGX_SHUTDOWN_SIGNAL</literal> signal. In single-process mode, nginx creates a new cycle, but keeps the old one until there are no longer clients with active connections tied to it. The worker and helper processes ignore this signal. </para> </listitem> <listitem> <para> <literal>NGX_REOPEN_SIGNAL</literal> (<literal>SIGUSR1</literal> on most systems) — Reopen files. The master process sends this signal to workers, which reopen all <literal>open_files</literal> related to the cycle. </para> </listitem> <listitem> <para> <literal>NGX_CHANGEBIN_SIGNAL</literal> (<literal>SIGUSR2</literal> on most systems) — Change the nginx binary. The master process starts a new nginx binary and passes in a list of all listen sockets. The text-format list, passed in the <literal>“NGINX”</literal> environment variable, consists of descriptor numbers separated with semicolons. The new nginx binary reads the <literal>“NGINX”</literal> variable and adds the sockets to its init cycle. Other processes ignore this signal. </para> </listitem> </list> <para> While all nginx worker processes are able to receive and properly handle POSIX signals, the master process does not use the standard <literal>kill()</literal> syscall to pass signals to workers and helpers. Instead, nginx uses inter-process socket pairs which allow sending messages between all nginx processes. Currently, however, messages are only sent from the master to its children. The messages carry the standard signals. </para> </section> <section name="Threads" id="threads"> <para> It is possible to offload into a separate thread tasks that would otherwise block the nginx worker process. For example, nginx can be configured to use threads to perform <link doc="../http/ngx_http_core_module.xml" id="aio">file I/O</link>. Another use case is a library that doesn't have asynchronous interface and thus cannot be normally used with nginx. Keep in mind that the threads interface is a helper for the existing asynchronous approach to processing client connections, and by no means intended as a replacement. </para> <para> To deal with synchronization, the following wrappers over <literal>pthreads</literal> primitives are available: <list type="bullet"> <listitem> <literal>typedef pthread_mutex_t ngx_thread_mutex_t;</literal> <list type="bullet"> <listitem> <literal>ngx_int_t ngx_thread_mutex_create(ngx_thread_mutex_t *mtx, ngx_log_t *log);</literal> </listitem> <listitem> <literal>ngx_int_t ngx_thread_mutex_destroy(ngx_thread_mutex_t *mtx, ngx_log_t *log);</literal> </listitem> <listitem> <literal>ngx_int_t ngx_thread_mutex_lock(ngx_thread_mutex_t *mtx, ngx_log_t *log);</literal> </listitem> <listitem> <literal>ngx_int_t ngx_thread_mutex_unlock(ngx_thread_mutex_t *mtx, ngx_log_t *log);</literal> </listitem> </list> </listitem> <listitem> <literal>typedef pthread_cond_t ngx_thread_cond_t;</literal> <list type="bullet"> <listitem> <literal>ngx_int_t ngx_thread_cond_create(ngx_thread_cond_t *cond, ngx_log_t *log);</literal> </listitem> <listitem> <literal>ngx_int_t ngx_thread_cond_destroy(ngx_thread_cond_t *cond, ngx_log_t *log);</literal> </listitem> <listitem> <literal>ngx_int_t ngx_thread_cond_signal(ngx_thread_cond_t *cond, ngx_log_t *log);</literal> </listitem> <listitem> <literal>ngx_int_t ngx_thread_cond_wait(ngx_thread_cond_t *cond, ngx_thread_mutex_t *mtx, ngx_log_t *log);</literal> </listitem> </list> </listitem> </list> </para> <para> Instead of creating a new thread for each task, nginx implements a <link doc="../ngx_core_module.xml" id="thread_pool"/> strategy. Multiple thread pools may be configured for different purposes (for example, performing I/O on different sets of disks). Each thread pool is created at startup and contains a limited number of threads that process a queue of tasks. When a task is completed, a predefined completion handler is called. </para> <para> The <literal>src/core/ngx_thread_pool.h</literal> header file contains relevant definitions: <programlisting> struct ngx_thread_task_s { ngx_thread_task_t *next; ngx_uint_t id; void *ctx; void (*handler)(void *data, ngx_log_t *log); ngx_event_t event; }; typedef struct ngx_thread_pool_s ngx_thread_pool_t; ngx_thread_pool_t *ngx_thread_pool_add(ngx_conf_t *cf, ngx_str_t *name); ngx_thread_pool_t *ngx_thread_pool_get(ngx_cycle_t *cycle, ngx_str_t *name); ngx_thread_task_t *ngx_thread_task_alloc(ngx_pool_t *pool, size_t size); ngx_int_t ngx_thread_task_post(ngx_thread_pool_t *tp, ngx_thread_task_t *task); </programlisting> At configuration time, a module willing to use threads has to obtain a reference to a thread pool by calling <literal>ngx_thread_pool_add(cf, name)</literal>, which either creates a new thread pool with the given <literal>name</literal> or returns a reference to the pool with that name if it already exists. </para> <para> To add a <literal>task</literal> into a queue of a specified thread pool <literal>tp</literal> at runtime, use the <literal>ngx_thread_task_post(tp, task)</literal> function. To execute a function in a thread, pass parameters and setup a completion handler using the <literal>ngx_thread_task_t</literal> structure: <programlisting> typedef struct { int foo; } my_thread_ctx_t; static void my_thread_func(void *data, ngx_log_t *log) { my_thread_ctx_t *ctx = data; /* this function is executed in a separate thread */ } static void my_thread_completion(ngx_event_t *ev) { my_thread_ctx_t *ctx = ev->data; /* executed in nginx event loop */ } ngx_int_t my_task_offload(my_conf_t *conf) { my_thread_ctx_t *ctx; ngx_thread_task_t *task; task = ngx_thread_task_alloc(conf->pool, sizeof(my_thread_ctx_t)); if (task == NULL) { return NGX_ERROR; } ctx = task->ctx; ctx->foo = 42; task->handler = my_thread_func; task->event.handler = my_thread_completion; task->event.data = ctx; if (ngx_thread_task_post(conf->thread_pool, task) != NGX_OK) { return NGX_ERROR; } return NGX_OK; } </programlisting> </para> </section> <section name="Modules" id="Modules"> <section name="Adding new modules" id="adding_new_modules"> <para> Each standalone nginx module resides in a separate directory that contains at least two files: <literal>config</literal> and a file with the module source code. The <literal>config</literal> file contains all information needed for nginx to integrate the module, for example: <programlisting> ngx_module_type=CORE ngx_module_name=ngx_foo_module ngx_module_srcs="$ngx_addon_dir/ngx_foo_module.c" . auto/module ngx_addon_name=$ngx_module_name </programlisting> The <literal>config</literal> file is a POSIX shell script that can set and access the following variables: <list type="bullet"> <listitem> <literal>ngx_module_type</literal> — Type of module to build. Possible values are <literal>CORE</literal>, <literal>HTTP</literal>, <literal>HTTP_FILTER</literal>, <literal>HTTP_INIT_FILTER</literal>, <literal>HTTP_AUX_FILTER</literal>, <literal>MAIL</literal>, <literal>STREAM</literal>, or <literal>MISC</literal>. </listitem> <listitem> <literal>ngx_module_name</literal> — Module names. To build multiple modules from a set of source files, specify a whitespace-separated list of names. The first name indicates the name of the output binary for the dynamic module. The names in the list must match the names used in the source code. </listitem> <listitem> <literal>ngx_addon_name</literal> — Name of the module as it appears in output on the console from the configure script. </listitem> <listitem> <literal>ngx_module_srcs</literal> — Whitespace-separated list of source files used to compile the module. The <literal>$ngx_addon_dir</literal> variable can be used to represent the path to the module directory. </listitem> <listitem> <literal>ngx_module_incs</literal> — Include paths required to build the module </listitem> <listitem> <literal>ngx_module_deps</literal> — Whitespace-separated list of the module's dependencies. Usually, it is the list of header files. </listitem> <listitem> <literal>ngx_module_libs</literal> — Whitespace-separated list of libraries to link with the module. For example, use <literal>ngx_module_libs=-lpthread</literal> to link <literal>libpthread</literal> library. The following macros can be used to link against the same libraries as nginx: <literal>LIBXSLT</literal>, <literal>LIBGD</literal>, <literal>GEOIP</literal>, <literal>PCRE</literal>, <literal>OPENSSL</literal>, <literal>MD5</literal>, <literal>SHA1</literal>, <literal>ZLIB</literal>, and <literal>PERL</literal>. </listitem> <listitem> <literal>ngx_module_link</literal> — Variable set by the build system to <literal>DYNAMIC</literal> for a dynamic module or <literal>ADDON</literal> for a static module and used to determine different actions to perform depending on linking type. </listitem> <listitem> <literal>ngx_module_order</literal> — Load order for the module; useful for the <literal>HTTP_FILTER</literal> and <literal>HTTP_AUX_FILTER</literal> module types. The format for this option is a whitespace-separated list of modules. All modules in the list following the current module's name end up after it in the global list of modules, which sets up the order for modules initialization. For filter modules later initialization means earlier execution. <para> The following modules are typically used as references. The <literal>ngx_http_copy_filter_module</literal> reads the data for other filter modules and is placed near the bottom of the list so that it is one of the first to be executed. The <literal>ngx_http_write_filter_module</literal> writes the data to the client socket and is placed near the top of the list, and is the last to be executed. </para> <para> By default, filter modules are placed before the <literal>ngx_http_copy_filter</literal> in the module list so that the filter handler is executed after the copy filter handler. For other module types the default is the empty string. </para> </listitem> </list> To compile a module into nginx statically, use the <literal>--add-module=/path/to/module</literal> argument to the configure script. To compile a module for later dynamic loading into nginx, use the <literal>--add-dynamic-module=/path/to/module</literal> argument. </para> </section> <section name="Core Modules" id="core_modules"> <para> Modules are the building blocks of nginx, and most of its functionality is implemented as modules. The module source file must contain a global variable of type <literal>ngx_module_t</literal>, which is defined as follows: <programlisting> struct ngx_module_s { /* private part is omitted */ void *ctx; ngx_command_t *commands; ngx_uint_t type; ngx_int_t (*init_master)(ngx_log_t *log); ngx_int_t (*init_module)(ngx_cycle_t *cycle); ngx_int_t (*init_process)(ngx_cycle_t *cycle); ngx_int_t (*init_thread)(ngx_cycle_t *cycle); void (*exit_thread)(ngx_cycle_t *cycle); void (*exit_process)(ngx_cycle_t *cycle); void (*exit_master)(ngx_cycle_t *cycle); /* stubs for future extensions are omitted */ }; </programlisting> The omitted private part includes the module version and a signature and is filled using the predefined macro <literal>NGX_MODULE_V1</literal>. </para> <para> Each module keeps its private data in the <literal>ctx</literal> field, recognizes the configuration directives, specified in the <literal>commands</literal> array, and can be invoked at certain stages of nginx lifecycle. The module lifecycle consists of the following events: <list type="bullet"> <listitem> Configuration directive handlers are called as they appear in configuration files in the context of the master process. </listitem> <listitem> After the configuration is parsed successfully, <literal>init_module</literal> handler is called in the context of the master process. The <literal>init_module</literal> handler is called in the master process each time a configuration is loaded. </listitem> <listitem> The master process creates one or more worker processes and the <literal>init_process</literal> handler is called in each of them. </listitem> <listitem> When a worker process receives the shutdown or terminate command from the master, it invokes the <literal>exit_process</literal> handler. </listitem> <listitem> The master process calls the <literal>exit_master</literal> handler before exiting. </listitem> </list> Because threads are used in nginx only as a supplementary I/O facility with its own API, <literal>init_thread</literal> and <literal>exit_thread</literal> handlers are not currently called. There is also no <literal>init_master</literal> handler, because it would be unnecessary overhead. </para> <para> The module <literal>type</literal> defines exactly what is stored in the <literal>ctx</literal> field. Its value is one of the following types: <list type="bullet"> <listitem><literal>NGX_CORE_MODULE</literal></listitem> <listitem><literal>NGX_EVENT_MODULE</literal></listitem> <listitem><literal>NGX_HTTP_MODULE</literal></listitem> <listitem><literal>NGX_MAIL_MODULE</literal></listitem> <listitem><literal>NGX_STREAM_MODULE</literal></listitem> </list> The <literal>NGX_CORE_MODULE</literal> is the most basic and thus the most generic and most low-level type of module. The other module types are implemented on top of it and provide a more convenient way to deal with corresponding domains, like handling events or HTTP requests. </para> <para> The set of core modules includes <literal>ngx_core_module</literal>, <literal>ngx_errlog_module</literal>, <literal>ngx_regex_module</literal>, <literal>ngx_thread_pool_module</literal> and <literal>ngx_openssl_module</literal> modules. The HTTP module, the stream module, the mail module and event modules are core modules too. The context of a core module is defined as: <programlisting> typedef struct { ngx_str_t name; void *(*create_conf)(ngx_cycle_t *cycle); char *(*init_conf)(ngx_cycle_t *cycle, void *conf); } ngx_core_module_t; </programlisting> where the <literal>name</literal> is a module name string, <literal>create_conf</literal> and <literal>init_conf</literal> are pointers to functions that create and initialize module configuration respectively. For core modules, nginx calls <literal>create_conf</literal> before parsing a new configuration and <literal>init_conf</literal> after all configuration is parsed successfully. The typical <literal>create_conf</literal> function allocates memory for the configuration and sets default values. </para> <para> For example, a simplistic module called <literal>ngx_foo_module</literal> might look like this: <programlisting> /* * Copyright (C) Author. */ #include <ngx_config.h> #include <ngx_core.h> typedef struct { ngx_flag_t enable; } ngx_foo_conf_t; static void *ngx_foo_create_conf(ngx_cycle_t *cycle); static char *ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf); static char *ngx_foo_enable(ngx_conf_t *cf, void *post, void *data); static ngx_conf_post_t ngx_foo_enable_post = { ngx_foo_enable }; static ngx_command_t ngx_foo_commands[] = { { ngx_string("foo_enabled"), NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, 0, offsetof(ngx_foo_conf_t, enable), &ngx_foo_enable_post }, ngx_null_command }; static ngx_core_module_t ngx_foo_module_ctx = { ngx_string("foo"), ngx_foo_create_conf, ngx_foo_init_conf }; ngx_module_t ngx_foo_module = { NGX_MODULE_V1, &ngx_foo_module_ctx, /* module context */ ngx_foo_commands, /* module directives */ NGX_CORE_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static void * ngx_foo_create_conf(ngx_cycle_t *cycle) { ngx_foo_conf_t *fcf; fcf = ngx_pcalloc(cycle->pool, sizeof(ngx_foo_conf_t)); if (fcf == NULL) { return NULL; } fcf->enable = NGX_CONF_UNSET; return fcf; } static char * ngx_foo_init_conf(ngx_cycle_t *cycle, void *conf) { ngx_foo_conf_t *fcf = conf; ngx_conf_init_value(fcf->enable, 0); return NGX_CONF_OK; } static char * ngx_foo_enable(ngx_conf_t *cf, void *post, void *data) { ngx_flag_t *fp = data; if (*fp == 0) { return NGX_CONF_OK; } ngx_log_error(NGX_LOG_NOTICE, cf->log, 0, "Foo Module is enabled"); return NGX_CONF_OK; } </programlisting> </para> </section> <section name="Configuration Directives" id="config_directives"> <para> The <literal>ngx_command_t</literal> type defines a single configuration directive. Each module that supports configuration provides an array of such structures that describe how to process arguments and what handlers to call: <programlisting> typedef struct ngx_command_s ngx_command_t; struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; }; </programlisting> Terminate the array with the special value <literal>ngx_null_command</literal>. The <literal>name</literal> is the name of a directive as it appears in the configuration file, for example "worker_processes" or "listen". The <literal>type</literal> is a bit-field of flags that specify the number of arguments the directive takes, its type, and the context in which it appears. The flags are: <list type="bullet"> <listitem> <literal>NGX_CONF_NOARGS</literal> — Directive takes no arguments. </listitem> <listitem> <literal>NGX_CONF_1MORE</literal> — Directive takes one or more arguments. </listitem> <listitem> <literal>NGX_CONF_2MORE</literal> — Directive takes two or more arguments. </listitem> <listitem> <literal>NGX_CONF_TAKE1</literal>..<literal>NGX_CONF_TAKE7</literal> — Directive takes exactly the indicated number of arguments. </listitem> <listitem> <literal>NGX_CONF_TAKE12</literal>, <literal>NGX_CONF_TAKE13</literal>, <literal>NGX_CONF_TAKE23</literal>, <literal>NGX_CONF_TAKE123</literal>, <literal>NGX_CONF_TAKE1234</literal> — Directive may take different number of arguments. Options are limited to the given numbers. For example, <literal>NGX_CONF_TAKE12</literal> means it takes one or two arguments. </listitem> </list> The flags for directive types are: <list type="bullet"> <listitem> <literal>NGX_CONF_BLOCK</literal> — Directive is a block, that is, it can contain other directives within its opening and closing braces, or even implement its own parser to handle contents inside. </listitem> <listitem> <literal>NGX_CONF_FLAG</literal> — Directive takes a boolean value, either <literal>on</literal> or <literal>off</literal>. </listitem> </list> A directive's context defines where it may appear in the configuration: <list type="bullet"> <listitem> <literal>NGX_MAIN_CONF</literal> — In the top level context. </listitem> <listitem> <literal>NGX_HTTP_MAIN_CONF</literal> — In the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_SRV_CONF</literal> — In a <literal>server</literal> block within the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_LOC_CONF</literal> — In a <literal>location</literal> block within the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_UPS_CONF</literal> — In an <literal>upstream</literal> block within the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_SIF_CONF</literal> — In an <literal>if</literal> block within a <literal>server</literal> block in the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_LIF_CONF</literal> — In an <literal>if</literal> block within a <literal>location</literal> block in the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_LMT_CONF</literal> — In a <literal>limit_except</literal> block within the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_STREAM_MAIN_CONF</literal> — In the <literal>stream</literal> block. </listitem> <listitem> <literal>NGX_STREAM_SRV_CONF</literal> — In a <literal>server</literal> block within the <literal>stream</literal> block. </listitem> <listitem> <literal>NGX_STREAM_UPS_CONF</literal> — In an <literal>upstream</literal> block within the <literal>stream</literal> block. </listitem> <listitem> <literal>NGX_MAIL_MAIN_CONF</literal> — In the <literal>mail</literal> block. </listitem> <listitem> <literal>NGX_MAIL_SRV_CONF</literal> — In a <literal>server</literal> block within the <literal>mail</literal> block. </listitem> <listitem> <literal>NGX_EVENT_CONF</literal> — In the <literal>event</literal> block. </listitem> <listitem> <literal>NGX_DIRECT_CONF</literal> — Used by modules that don't create a hierarchy of contexts and only have one global configuration. This configuration is passed to the handler as the <literal>conf</literal> argument. </listitem> </list> The configuration parser uses these flags to throw an error in case of a misplaced directive and calls directive handlers supplied with a proper configuration pointer, so that the same directives in different locations can store their values in distinct places. </para> <para> The <literal>set</literal> field defines a handler that processes a directive and stores parsed values into the corresponding configuration. There's a number of functions that perform common conversions: <list type="bullet"> <listitem> <literal>ngx_conf_set_flag_slot</literal> — Converts the literal strings <literal>on</literal> and <literal>off</literal> into an <literal>ngx_flag_t</literal> value with values 1 or 0, respectively. </listitem> <listitem> <literal>ngx_conf_set_str_slot</literal> — Stores a string as a value of the <literal>ngx_str_t</literal> type. </listitem> <listitem> <literal>ngx_conf_set_str_array_slot</literal> — Appends a value to an array <literal>ngx_array_t</literal> of strings <literal>ngx_str_t</literal>. The array is created if does not already exist. </listitem> <listitem> <literal>ngx_conf_set_keyval_slot</literal> — Appends a key-value pair to an array <literal>ngx_array_t</literal> of key-value pairs <literal>ngx_keyval_t</literal>. The first string becomes the key and the second the value. The array is created if it does not already exist. </listitem> <listitem> <literal>ngx_conf_set_num_slot</literal> — Converts a directive's argument to an <literal>ngx_int_t</literal> value. </listitem> <listitem> <literal>ngx_conf_set_size_slot</literal> — Converts a <link doc="../syntax.xml">size</link> to a <literal>size_t</literal> value expressed in bytes. </listitem> <listitem> <literal>ngx_conf_set_off_slot</literal> — Converts an <link doc="../syntax.xml">offset</link> to an <literal>off_t</literal> value expressed in bytes. </listitem> <listitem> <literal>ngx_conf_set_msec_slot</literal> — Converts a <link doc="../syntax.xml">time</link> to an <literal>ngx_msec_t</literal> value expressed in milliseconds. </listitem> <listitem> <literal>ngx_conf_set_sec_slot</literal> — Converts a <link doc="../syntax.xml">time</link> to a <literal>time_t</literal> value expressed in in seconds. </listitem> <listitem> <literal>ngx_conf_set_bufs_slot</literal> — Converts the two supplied arguments into an <literal>ngx_bufs_t</literal> object that holds the number and <link doc="../syntax.xml">size</link> of buffers. </listitem> <listitem> <literal>ngx_conf_set_enum_slot</literal> — Converts the supplied argument into an <literal>ngx_uint_t</literal> value. The null-terminated array of <literal>ngx_conf_enum_t</literal> passed in the <literal>post</literal> field defines the acceptable strings and corresponding integer values. </listitem> <listitem> <literal>ngx_conf_set_bitmask_slot</literal> — Converts the supplied arguments into an <literal>ngx_uint_t</literal> value. The mask values for each argument are ORed producing the result. The null-terminated array of <literal>ngx_conf_bitmask_t</literal> passed in the <literal>post</literal> field defines the acceptable strings and corresponding mask values. </listitem> <listitem> <literal>set_path_slot</literal> — Converts the supplied arguments to an <literal>ngx_path_t</literal> value and performs all required initializations. For details, see the documentation for the <link doc="../http/ngx_http_proxy_module.xml" id="proxy_temp_path"> proxy_temp_path</link> directive. </listitem> <listitem> <literal>set_access_slot</literal> — Converts the supplied arguments to a file permissions mask. For details, see the documentation for the <link doc="../http/ngx_http_proxy_module.xml" id="proxy_store_access"> proxy_store_access</link> directive. </listitem> </list> </para> <para> The <literal>conf</literal> field defines which configuration structure is passed to the directory handler. Core modules only have the global configuration and set <literal>NGX_DIRECT_CONF</literal> flag to access it. Modules like HTTP, Stream or Mail create hierarchies of configurations. For example, a module's configuration is created for <literal>server</literal>, <literal>location</literal> and <literal>if</literal> scopes. <list type="bullet"> <listitem> <literal>NGX_HTTP_MAIN_CONF_OFFSET</literal> — Configuration for the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_SRV_CONF_OFFSET</literal> — Configuration for a <literal>server</literal> block within the <literal>http</literal> block. </listitem> <listitem> <literal>NGX_HTTP_LOC_CONF_OFFSET</literal> — Configuration for a <literal>location</literal> block within the <literal>http</literal>. </listitem> <listitem> <literal>NGX_STREAM_MAIN_CONF_OFFSET</literal> — Configuration for the <literal>stream</literal> block. </listitem> <listitem> <literal>NGX_STREAM_SRV_CONF_OFFSET</literal> — Configuration for a <literal>server</literal> block within the <literal>stream</literal> block. </listitem> <listitem> <literal>NGX_MAIL_MAIN_CONF_OFFSET</literal> — Configuration for the <literal>mail</literal> block. </listitem> <listitem> <literal>NGX_MAIL_SRV_CONF_OFFSET</literal> — Configuration for a <literal>server</literal> block within the <literal>mail</literal> block. </listitem> </list> </para> <para> The <literal>offset</literal> defines the offset of a field in a module configuration structure that holds values for this particular directive. The typical use is to employ the <literal>offsetof()</literal> macro. </para> <para> The <literal>post</literal> field has two purposes: it may be used to define a handler to be called after the main handler has completed, or to pass additional data to the main handler. In the first case, the <literal>ngx_conf_post_t</literal> structure needs to be initialized with a pointer to the handler, for example: <programlisting> static char *ngx_do_foo(ngx_conf_t *cf, void *post, void *data); static ngx_conf_post_t ngx_foo_post = { ngx_do_foo }; </programlisting> The <literal>post</literal> argument is the <literal>ngx_conf_post_t</literal> object itself, and the <literal>data</literal> is a pointer to the value, converted from arguments by the main handler with the appropriate type. </para> </section> </section> <section name="HTTP" id="http"> <section name="Connection" id="http_connection"> <para> Each HTTP client connection runs through the following stages: </para> <list type="bullet"> <listitem> <literal>ngx_event_accept()</literal> accepts a client TCP connection. This handler is called in response to a read notification on a listen socket. A new <literal>ngx_connection_t</literal> object is created at this stage to wrap the newly accepted client socket. Each nginx listener provides a handler to pass the new connection object to. For HTTP connections it's <literal>ngx_http_init_connection(c)</literal>. </listitem> <listitem> <literal>ngx_http_init_connection()</literal> performs early initialization of the HTTP connection. At this stage an <literal>ngx_http_connection_t</literal> object is created for the connection and its reference is stored in the connection's <literal>data</literal> field. Later it will be replaced by an HTTP request object. A PROXY protocol parser and the SSL handshake are started at this stage as well. </listitem> <listitem> <literal>ngx_http_wait_request_handler()</literal> read event handler is called when data is available on the client socket. At this stage an HTTP request object <literal>ngx_http_request_t</literal> is created and set to the connection's <literal>data</literal> field. </listitem> <listitem> <literal>ngx_http_process_request_line()</literal> read event handler reads client request line. The handler is set by <literal>ngx_http_wait_request_handler()</literal>. The data is read into connection's <literal>buffer</literal>. The size of the buffer is initially set by the directive <link doc="../http/ngx_http_core_module.xml" id="client_header_buffer_size"/>. The entire client header is supposed to fit in the buffer. If the initial size is not sufficient, a bigger buffer is allocated, with the capacity set by the <literal>large_client_header_buffers</literal> directive. </listitem> <listitem> <literal>ngx_http_process_request_headers()</literal> read event handler, is set after <literal>ngx_http_process_request_line()</literal> to read the client request header. </listitem> <listitem> <literal>ngx_http_core_run_phases()</literal> is called when the request header is completely read and parsed. This function runs request phases from <literal>NGX_HTTP_POST_READ_PHASE</literal> to <literal>NGX_HTTP_CONTENT_PHASE</literal>. The last phase is intended to generate a response and pass it along the filter chain. The response is not necessarily sent to the client at this phase. It might remain buffered and be sent at the finalization stage. </listitem> <listitem> <literal>ngx_http_finalize_request()</literal> is usually called when the request has generated all the output or produced an error. In the latter case an appropriate error page is looked up and used as the response. If the response is not completely sent to the client by this point, an HTTP writer <literal>ngx_http_writer()</literal> is activated to finish sending outstanding data. </listitem> <listitem> <literal>ngx_http_finalize_connection()</literal> is called when the complete response has been sent to the client and the request can be destroyed. If the client connection keepalive feature is enabled, <literal>ngx_http_set_keepalive()</literal> is called, which destroys the current request and waits for the next request on the connection. Otherwise, <literal>ngx_http_close_request()</literal> destroys both the request and the connection. </listitem> </list> </section> <section name="Request" id="http_request"> <para> For each client HTTP request the <literal>ngx_http_request_t</literal> object is created. Some of the fields of this object are: </para> <list type="bullet"> <listitem> <para> <literal>connection</literal> — Pointer to a <literal>ngx_connection_t</literal> client connection object. Several requests can reference the same connection object at the same time - one main request and its subrequests. After a request is deleted, a new request can be created on the same connection. </para> <para> Note that for HTTP connections <literal>ngx_connection_t</literal>'s <literal>data</literal> field points back to the request. Such requests are called active, as opposed to the other requests tied to the connection. An active request is used to handle client connection events and is allowed to output its response to the client. Normally, each request becomes active at some point so that it can send its output. </para> </listitem> <listitem> <para> <literal>ctx</literal> — Array of HTTP module contexts. Each module of type <literal>NGX_HTTP_MODULE</literal> can store any value (normally, a pointer to a structure) in the request. The value is stored in the <literal>ctx</literal> array at the module's <literal>ctx_index</literal> position. The following macros provide a convenient way to get and set request contexts: </para> <list type="bullet"> <listitem> <literal>ngx_http_get_module_ctx(r, module)</literal> — Returns the <literal>module</literal>'s context </listitem> <listitem> <literal>ngx_http_set_ctx(r, c, module)</literal> — Sets <literal>c</literal> as the <literal>module</literal>'s context </listitem> </list> </listitem> <listitem> <literal>main_conf</literal>, <literal>srv_conf</literal>, <literal>loc_conf</literal> — Arrays of current request configurations. Configurations are stored at the module's <literal>ctx_index</literal> positions. </listitem> <listitem> <literal>read_event_handler</literal>, <literal>write_event_handler</literal> - Read and write event handlers for the request. Normally, both the read and write event handlers for an HTTP connection are set to <literal>ngx_http_request_handler()</literal>. This function calls the <literal>read_event_handler</literal> and <literal>write_event_handler</literal> handlers for the currently active request. </listitem> <listitem> <literal>cache</literal> — Request cache object for caching the upstream response. </listitem> <listitem> <literal>upstream</literal> — Request upstream object for proxying. </listitem> <listitem> <literal>pool</literal> — Request pool. The request object itself is allocated in this pool, which is destroyed when the request is deleted. For allocations that need to be available throughout the client connection's lifetime, use <literal>ngx_connection_t</literal>'s pool instead. </listitem> <listitem> <literal>header_in</literal> — Buffer into which the client HTTP request header is read. </listitem> <listitem> <literal>headers_in</literal>, <literal>headers_out</literal> — Input and output HTTP headers objects. Both objects contain the <literal>headers</literal> field of type <literal>ngx_list_t</literal> for keeping the raw list of headers. In addition to that, specific headers are available for getting and setting as separate fields, for example <literal>content_length_n</literal>, <literal>status</literal> etc. </listitem> <listitem> <literal>request_body</literal> — Client request body object. </listitem> <listitem> <literal>start_sec</literal>, <literal>start_msec</literal> — Time point when the request was created, used for tracking request duration. </listitem> <listitem> <literal>method</literal>, <literal>method_name</literal> — Numeric and text representation of the client HTTP request method. Numeric values for methods are defined in <literal>src/http/ngx_http_request.h</literal> with the macros <literal>NGX_HTTP_GET</literal>, <literal>NGX_HTTP_HEAD</literal>, <literal>NGX_HTTP_POST</literal>, etc. </listitem> <listitem> <literal>http_protocol</literal> — Client HTTP protocol version in its original text form (“HTTP/1.0”, “HTTP/1.1” etc). </listitem> <listitem> <literal>http_version</literal> — Client HTTP protocol version in numeric form (<literal>NGX_HTTP_VERSION_10</literal>, <literal>NGX_HTTP_VERSION_11</literal>, etc.). </listitem> <listitem> <literal>http_major</literal>, <literal>http_minor</literal> — Client HTTP protocol version in numeric form split into major and minor parts. </listitem> <listitem> <literal>request_line</literal>, <literal>unparsed_uri</literal> — Request line and URI in the original client request. </listitem> <listitem> <literal>uri</literal>, <literal>args</literal>, <literal>exten</literal> — URI, arguments and file extension for the current request. The URI value here might differ from the original URI sent by the client due to normalization. Throughout request processing, these values can change as internal redirects are performed. </listitem> <listitem> <literal>main</literal> — Pointer to a main request object. This object is created to process a client HTTP request, as opposed to subrequests, which are created to perform a specific subtask within the main request. </listitem> <listitem> <literal>parent</literal> — Pointer to the parent request of a subrequest. </listitem> <listitem> <literal>postponed</literal> — List of output buffers and subrequests, in the order in which they are sent and created. The list is used by the postpone filter to provide consistent request output when parts of it are created by subrequests. </listitem> <listitem> <literal>post_subrequest</literal> — Pointer to a handler with the context to be called when a subrequest gets finalized. Unused for main requests. </listitem> <listitem> <para> <literal>posted_requests</literal> — List of requests to be started or resumed, which is done by calling the request's <literal>write_event_handler</literal>. Normally, this handler holds the request main function, which at first runs request phases and then produces the output. </para> <para> A request is usually posted by the <literal>ngx_http_post_request(r, NULL)</literal> call. It is always posted to the main request <literal>posted_requests</literal> list. The function <literal>ngx_http_run_posted_requests(c)</literal> runs all requests that are posted in the main request of the passed connection's active request. All event handlers call <literal>ngx_http_run_posted_requests</literal>, which can lead to new posted requests. Normally, it is called after invoking a request's read or write handler. </para> </listitem> <listitem> <literal>phase_handler</literal> — Index of current request phase. </listitem> <listitem> <literal>ncaptures</literal>, <literal>captures</literal>, <literal>captures_data</literal> — Regex captures produced by the last regex match of the request. A regex match can occur at a number of places during request processing: map lookup, server lookup by SNI or HTTP Host, rewrite, proxy_redirect, etc. Captures produced by a lookup are stored in the above mentioned fields. The field <literal>ncaptures</literal> holds the number of captures, <literal>captures</literal> holds captures boundaries and <literal>captures_data</literal> holds the string against which the regex was matched and which is used to extract captures. After each new regex match, request captures are reset to hold new values. </listitem> <listitem> <literal>count</literal> — Request reference counter. The field only makes sense for the main request. Increasing the counter is done by simple <literal>r->main->count++</literal>. To decrease the counter, call <literal>ngx_http_finalize_request(r, rc)</literal>. Creating of a subrequest and running the request body read process both increment the counter. </listitem> <listitem> <literal>subrequests</literal> — Current subrequest nesting level. Each subrequest inherits its parent's nesting level, decreased by one. An error is generated if the value reaches zero. The value for the main request is defined by the <literal>NGX_HTTP_MAX_SUBREQUESTS</literal> constant. </listitem> <listitem> <literal>uri_changes</literal> — Number of URI changes remaining for the request. The total number of times a request can change its URI is limited by the <literal>NGX_HTTP_MAX_URI_CHANGES</literal> constant. With each change the value is decremented until it reaches zero, at which time an error is generated. Rewrites and internal redirects to normal or named locations are considered URI changes. </listitem> <listitem> <literal>blocked</literal> — Counter of blocks held on the request. While this value is non-zero, the request cannot be terminated. Currently, this value is increased by pending AIO operations (POSIX AIO and thread operations) and active cache lock. </listitem> <listitem> <literal>buffered</literal> — Bitmask showing which modules have buffered the output produced by the request. A number of filters can buffer output; for example, sub_filter can buffer data because of a partial string match, copy filter can buffer data because of the lack of free output buffers etc. As long as this value is non-zero, the request is not finalized pending the flush. </listitem> <listitem> <literal>header_only</literal> — Flag indicating that the output does not require a body. For example, this flag is used by HTTP HEAD requests. </listitem> <listitem> <para> <literal>keepalive</literal> — Flag indicating whether client connection keepalive is supported. The value is inferred from the HTTP version and the value of the <header>Connection</header> header. </para> </listitem> <listitem> <literal>header_sent</literal> — Flag indicating that the output header has already been sent by the request. </listitem> <listitem> <literal>internal</literal> — Flag indicating that the current request is internal. To enter the internal state, a request must pass through an internal redirect or be a subrequest. Internal requests are allowed to enter internal locations. </listitem> <listitem> <literal>allow_ranges</literal> — Flag indicating that a partial response can be sent to the client, as requested by the HTTP Range header. </listitem> <listitem> <literal>subrequest_ranges</literal> — Flag indicating that a partial response can be sent while a subrequest is being processed. </listitem> <listitem> <literal>single_range</literal> — Flag indicating that only a single continuous range of output data can be sent to the client. This flag is usually set when sending a stream of data, for example from a proxied server, and the entire response is not available in one buffer. </listitem> <listitem> <literal>main_filter_need_in_memory</literal>, <literal>filter_need_in_memory</literal> — Flags requesting that the output produced in memory buffers rather than files. This is a signal to the copy filter to read data from file buffers even if sendfile is enabled. The difference between the two flags is the location of the filter modules that set them. Filters called before the postpone filter in the filter chain set <literal>filter_need_in_memory</literal>, requesting that only the current request output come in memory buffers. Filters called later in the filter chain set <literal>main_filter_need_in_memory</literal>, requesting that both the main request and all subrequests read files in memory while sending output. </listitem> <listitem> <literal>filter_need_temporary</literal> — Flag requesting that the request output be produced in temporary buffers, but not in readonly memory buffers or file buffers. This is used by filters which may change output directly in the buffers where it's sent.</listitem> </list> </section> <section name="Configuration" id="http_conf"> <para> Each HTTP module can have three types of configuration: </para> <list type="bullet"> <listitem> Main configuration — Applies to the entire <literal>http</literal> block. Functions as global settings for a module. </listitem> <listitem> Server configuration — Applies to a single <literal>server</literal> block. Functions as server-specific settings for a module. </listitem> <listitem> Location configuration — Applies to a single <literal>location</literal>, <literal>if</literal> or <literal>limit_except</literal> block. Functions as location-specific settings for a module. </listitem> </list> <para> Configuration structures are created at the nginx configuration stage by calling functions, which allocate the structures, initialize them and merge them. The following example shows how to create a simple location configuration for a module. The configuration has one setting, <literal>foo</literal>, of type unsigned integer. </para> <programlisting> typedef struct { ngx_uint_t foo; } ngx_http_foo_loc_conf_t; static ngx_http_module_t ngx_http_foo_module_ctx = { NULL, /* preconfiguration */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ ngx_http_foo_create_loc_conf, /* create location configuration */ ngx_http_foo_merge_loc_conf /* merge location configuration */ }; static void * ngx_http_foo_create_loc_conf(ngx_conf_t *cf) { ngx_http_foo_loc_conf_t *conf; conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_foo_loc_conf_t)); if (conf == NULL) { return NULL; } conf->foo = NGX_CONF_UNSET_UINT; return conf; } static char * ngx_http_foo_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) { ngx_http_foo_loc_conf_t *prev = parent; ngx_http_foo_loc_conf_t *conf = child; ngx_conf_merge_uint_value(conf->foo, prev->foo, 1); } </programlisting> <para> As seen in the example, the <literal>ngx_http_foo_create_loc_conf()</literal> function creates a new configuration structure, and <literal>ngx_http_foo_merge_loc_conf()</literal> merges a configuration with configuration from a higher level. In fact, server and location configuration do not exist only at the server and location levels, but are also created for all levels above them. Specifically, a server configuration is also created at the main level and location configurations are created at the main, server, and location levels. These configurations make it possible to specify server- and location-specific settings at any level of an nginx configuration file. Eventually configurations are merged down. A number of macros like <literal>NGX_CONF_UNSET</literal> and <literal>NGX_CONF_UNSET_UINT</literal> are provided for indicating a missing setting and ignoring it while merging. Standard nginx merge macros like <literal>ngx_conf_merge_value()</literal> and <literal>ngx_conf_merge_uint_value()</literal> provide a convenient way to merge a setting and set the default value if none of the configurations provided an explicit value. For complete list of macros for different types, see <literal>src/core/ngx_conf_file.h</literal>. </para> <para> The following macros are available. for accessing configuration for HTTP modules at configuration time. They all take <literal>ngx_conf_t</literal> reference as the first argument. </para> <list type="bullet"> <listitem> <literal>ngx_http_conf_get_module_main_conf(cf, module)</literal> </listitem> <listitem> <literal>ngx_http_conf_get_module_srv_conf(cf, module)</literal> </listitem> <listitem> <literal>ngx_http_conf_get_module_loc_conf(cf, module)</literal> </listitem> </list> <para> The following example gets a pointer to a location configuration of standard nginx core module <link doc="../http/ngx_http_core_module.xml">ngx_http_core_module</link> and replaces the location content handler kept in the <literal>handler</literal> field of the structure. </para> <programlisting> static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r); static ngx_command_t ngx_http_foo_commands[] = { { ngx_string("foo"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS, ngx_http_foo, 0, 0, NULL }, ngx_null_command }; static char * ngx_http_foo(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_bar_handler; return NGX_CONF_OK; } </programlisting> <para> The following macros are available for accessing configuration for HTTP modules at runtime. </para> <list type="bullet"> <listitem> <literal>ngx_http_get_module_main_conf(r, module)</literal> </listitem> <listitem> <literal>ngx_http_get_module_srv_conf(r, module)</literal> </listitem> <listitem> <literal>ngx_http_get_module_loc_conf(r, module)</literal> </listitem> </list> <para> These macros receive a reference to an HTTP request <literal>ngx_http_request_t</literal>. The main configuration of a request never changes. Server configuration can change from the default after the virtual server for the request is chosen. Location configuration selected for processing a request can change multiple times as a result of a rewrite operation or internal redirect. The following example shows how to access a module's HTTP configuration at runtime. </para> <programlisting> static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r) { ngx_http_foo_loc_conf_t *flcf; flcf = ngx_http_get_module_loc_conf(r, ngx_http_foo_module); ... } </programlisting> </section> <section name="Phases" id="http_phases"> <para> Each HTTP request passes through a sequence of phases. In each phase a distinct type of processing is performed on the request. Module-specific handlers can be registered in most phases, and many standard nginx modules register their phase handlers as a way to get called at a specific stage of request processing. Phases are processed successively and the phase handlers are called once the request reaches the phase. Following is the list of nginx HTTP phases. </para> <list type="bullet"> <listitem> <literal>NGX_HTTP_POST_READ_PHASE</literal> — First phase. The <link doc="../http/ngx_http_realip_module.xml">ngx_http_realip_module</link> registers its handler at this phase to enable substitution of client addresses before any other module is invoked. </listitem> <listitem> <literal>NGX_HTTP_SERVER_REWRITE_PHASE</literal> — Phase where rewrite directives defined in a <literal>server</literal> block (but outside a <literal>location</literal> block) are processed. The <link doc="../http/ngx_http_rewrite_module.xml">ngx_http_rewrite_module</link> installs its handler at this phase. </listitem> <listitem> <literal>NGX_HTTP_FIND_CONFIG_PHASE</literal> — Special phase where a location is chosen based on the request URI. Before this phase, the default location for the relevant virtual server is assigned to the request, and any module requesting a location configuration receives the configuration for the default server location. This phase assigns a new location to the request. No additional handlers can be registered at this phase. </listitem> <listitem> <literal>NGX_HTTP_REWRITE_PHASE</literal> — Same as <literal>NGX_HTTP_SERVER_REWRITE_PHASE</literal>, but for rewrite rules defined in the location, chosen in the previous phase. </listitem> <listitem> <literal>NGX_HTTP_POST_REWRITE_PHASE</literal> — Special phase where the request is redirected to a new location if its URI changed during a rewrite. This is implemented by the request going through the <literal>NGX_HTTP_FIND_CONFIG_PHASE</literal> again. No additional handlers can be registered at this phase. </listitem> <listitem> <literal>NGX_HTTP_PREACCESS_PHASE</literal> — A common phase for different types of handlers, not associated with access control. The standard nginx modules <link doc="../http/ngx_http_limit_conn_module.xml">ngx_http_limit_conn_module </link> and <link doc="../http/ngx_http_limit_req_module.xml"> ngx_http_limit_req_module</link> register their handlers at this phase. </listitem> <listitem> <literal>NGX_HTTP_ACCESS_PHASE</literal> — Phase where it is verified that the client is authorized to make the request. Standard nginx modules such as <link doc="../http/ngx_http_access_module.xml">ngx_http_access_module</link> and <link doc="../http/ngx_http_auth_basic_module.xml">ngx_http_auth_basic_module </link> register their handlers at this phase. By default the client must pass the authorization check of all handlers registered at this phase for the request to continue to the next phase. The <link doc="../http/ngx_http_core_module.xml" id="satisfy"/> directive, can be used to permit processing to continue if any of the phase handlers authorizes the client. </listitem> <listitem> <literal>NGX_HTTP_POST_ACCESS_PHASE</literal> — Special phase where the <link doc="../http/ngx_http_core_module.xml" id="satisfy">satisfy any</link> directive is processed. If some access phase handlers denied access and none explicitly allowed it, the request is finalized. No additional handlers can be registered at this phase. </listitem> <listitem> <literal>NGX_HTTP_PRECONTENT_PHASE</literal> — Phase for handlers to be called prior to generating content. Standard modules such as <link doc="../http/ngx_http_core_module.xml" id="try_files"> ngx_http_try_files_module</link> and <link doc="../http/ngx_http_mirror_module.xml">ngx_http_mirror_module</link> register their handlers at this phase. </listitem> <listitem> <literal>NGX_HTTP_CONTENT_PHASE</literal> — Phase where the response is normally generated. Multiple nginx standard modules register their handlers at this phase, including <link doc="../http/ngx_http_index_module.xml">ngx_http_index_module</link> or <literal>ngx_http_static_module</literal>. They are called sequentially until one of them produces the output. It's also possible to set content handlers on a per-location basis. If the <link doc="../http/ngx_http_core_module.xml">ngx_http_core_module</link>'s location configuration has <literal>handler</literal> set, it is called as the content handler and the handlers installed at this phase are ignored. </listitem> <listitem> <literal>NGX_HTTP_LOG_PHASE</literal> — Phase where request logging is performed. Currently, only the <link doc="../http/ngx_http_log_module.xml">ngx_http_log_module</link> registers its handler at this stage for access logging. Log phase handlers are called at the very end of request processing, right before freeing the request. </listitem> </list> <para> Following is the example of a preaccess phase handler. </para> <programlisting> static ngx_http_module_t ngx_http_foo_module_ctx = { NULL, /* preconfiguration */ ngx_http_foo_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; static ngx_int_t ngx_http_foo_handler(ngx_http_request_t *r) { ngx_str_t *ua; ua = r->headers_in->user_agent; if (ua == NULL) { return NGX_DECLINED; } /* reject requests with "User-Agent: foo" */ if (ua->value.len == 3 && ngx_strncmp(ua->value.data, "foo", 3) == 0) { return NGX_HTTP_FORBIDDEN; } return NGX_DECLINED; } static ngx_int_t ngx_http_foo_init(ngx_conf_t *cf) { ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_PREACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_foo_handler; return NGX_OK; } </programlisting> <para> Phase handlers are expected to return specific codes: </para> <list type="bullet"> <listitem> <literal>NGX_OK</literal> — Proceed to the next phase. </listitem> <listitem> <literal>NGX_DECLINED</literal> — Proceed to the next handler of the current phase. If the current handler is the last in the current phase, move to the next phase. </listitem> <listitem> <literal>NGX_AGAIN</literal>, <literal>NGX_DONE</literal> — Suspend phase handling until some future event which can be an asynchronous I/O operation or just a delay, for example. It is assumed, that phase handling will be resumed later by calling <literal>ngx_http_core_run_phases()</literal>. </listitem> <listitem> Any other value returned by the phase handler is treated as a request finalization code, in particular, an HTTP response code. The request is finalized with the code provided. </listitem> </list> <para> For some phases, return codes are treated in a slightly different way. At the content phase, any return code other that <literal>NGX_DECLINED</literal> is considered a finalization code. Any return code from the location content handlers is considered a finalization code. At the access phase, in <link doc="../http/ngx_http_core_module.xml" id="satisfy">satisfy any</link> mode, any return code other than <literal>NGX_OK</literal>, <literal>NGX_DECLINED</literal>, <literal>NGX_AGAIN</literal>, <literal>NGX_DONE</literal> is considered a denial. If no subsequent access handlers allow or deny access with a different code, the denial code will become the finalization code. </para> </section> <section name="Variables" id="http_variables"> <section name="Accessing existing variables" id="http_existing_variables"> <para> Variables can be referenced by index (this is the most common method) or name (see <link id="http_creating_variables">below</link>). The index is created at configuration stage, when a variable is added to the configuration. To obtain the variable index, use <literal>ngx_http_get_variable_index()</literal>: <programlisting> ngx_str_t name; /* ngx_string("foo") */ ngx_int_t index; index = ngx_http_get_variable_index(cf, &name); </programlisting> Here, <literal>cf</literal> is a pointer to nginx configuration and <literal>name</literal> points to a string containing the variable name. The function returns <literal>NGX_ERROR</literal> on error or a valid index otherwise, which is typically stored somewhere in the module's configuration for future use. </para> <para> All HTTP variables are evaluated in the context of a given HTTP request, and results are specific to and cached in that HTTP request. All functions that evaluate variables return the <literal>ngx_http_variable_value_t</literal> type, representing the variable value: <programlisting> typedef ngx_variable_value_t ngx_http_variable_value_t; typedef struct { unsigned len:28; unsigned valid:1; unsigned no_cacheable:1; unsigned not_found:1; unsigned escape:1; u_char *data; } ngx_variable_value_t; </programlisting> where: <list type="bullet"> <listitem> <literal>len</literal> — The length of the value </listitem> <listitem> <literal>data</literal> — The value itself </listitem> <listitem> <literal>valid</literal> — The value is valid </listitem> <listitem> <literal>not_found</literal> — The variable was not found and thus the <literal>data</literal> and <literal>len</literal> fields are irrelevant; this can happen, for example, with variables like <var>$arg_foo</var> when a corresponding argument was not passed in a request </listitem> <listitem> <literal>no_cacheable</literal> — Do not cache result </listitem> <listitem> <literal>escape</literal> — Used internally by the logging module to mark values that require escaping on output. </listitem> </list> </para> <para> The <literal>ngx_http_get_flushed_variable()</literal> and <literal>ngx_http_get_indexed_variable()</literal> functions are used to obtain the value of a variable. They have the same interface - accepting an HTTP request <literal>r</literal> as a context for evaluating the variable and an <literal>index</literal> that identifies it. An example of typical usage: <programlisting> ngx_http_variable_value_t *v; v = ngx_http_get_flushed_variable(r, index); if (v == NULL || v->not_found) { /* we failed to get value or there is no such variable, handle it */ return NGX_ERROR; } /* some meaningful value is found */ </programlisting> The difference between functions is that the <literal>ngx_http_get_indexed_variable()</literal> returns a cached value and <literal>ngx_http_get_flushed_variable()</literal> flushes the cache for non-cacheable variables. </para> <para> Some modules, such as SSI and Perl, need to deal with variables for which the name is not known at configuration time. An index therefore cannot be used to access them, but the <literal>ngx_http_get_variable(r, name, key)</literal> function is available. It searches for a variable with a given <literal>name</literal> and its hash <literal>key</literal> derived from the name. </para> </section> <section name="Creating variables" id="http_creating_variables"> <para> To create a variable, use the <literal>ngx_http_add_variable()</literal> function. It takes as arguments a configuration (where the variable is registered), the variable name and flags that control the function's behaviour: <list type="bullet"> <listitem><literal>NGX_HTTP_VAR_CHANGEABLE</literal> — Enables redefinition of the variable: there is no conflict if another module defines a variable with the same name. This allows the <link doc="../http/ngx_http_rewrite_module.xml" id="set"/> directive to override variables. </listitem> <listitem><literal>NGX_HTTP_VAR_NOCACHEABLE</literal> — Disables caching, which is useful for variables such as <literal>$time_local</literal>. </listitem> <listitem><literal>NGX_HTTP_VAR_NOHASH</literal> — Indicates that this variable is only accessible by index, not by name. This is a small optimization for use when it is known that the variable is not needed in modules like SSI or Perl. </listitem> <listitem><literal>NGX_HTTP_VAR_PREFIX</literal> — The name of the variable is a prefix. In this case, a handler must implement additional logic to obtain the value of a specific variable. For example, all “<literal>arg_</literal>” variables are processed by the same handler, which performs lookup in request arguments and returns the value of a specific argument. </listitem> </list> The function returns NULL in case of error or a pointer to <literal>ngx_http_variable_t</literal> otherwise: <programlisting> struct ngx_http_variable_s { ngx_str_t name; ngx_http_set_variable_pt set_handler; ngx_http_get_variable_pt get_handler; uintptr_t data; ngx_uint_t flags; ngx_uint_t index; }; </programlisting> The <literal>get</literal> and <literal>set</literal> handlers are called to obtain or set the variable value, <literal>data</literal> is passed to variable handlers, and <literal>index</literal> holds assigned variable index used to reference the variable. </para> <para> Usually, a null-terminated static array of <literal>ngx_http_variable_t</literal> structures is created by a module and processed at the preconfiguration stage to add variables into the configuration, for example: <programlisting> static ngx_http_variable_t ngx_http_foo_vars[] = { { ngx_string("foo_v1"), NULL, ngx_http_foo_v1_variable, 0, 0, 0 }, ngx_http_null_variable }; static ngx_int_t ngx_http_foo_add_variables(ngx_conf_t *cf) { ngx_http_variable_t *var, *v; for (v = ngx_http_foo_vars; v->name.len; v++) { var = ngx_http_add_variable(cf, &v->name, v->flags); if (var == NULL) { return NGX_ERROR; } var->get_handler = v->get_handler; var->data = v->data; } return NGX_OK; } </programlisting> This function in the example is used to initialize the <literal>preconfiguration</literal> field of the HTTP module context and is called before the parsing of HTTP configuration, so that the parser can refer to these variables. </para> <para> The <literal>get</literal> handler is responsible for evaluating a variable in the context of a specific request, for example: <programlisting> static ngx_int_t ngx_http_variable_connection(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { u_char *p; p = ngx_pnalloc(r->pool, NGX_ATOMIC_T_LEN); if (p == NULL) { return NGX_ERROR; } v->len = ngx_sprintf(p, "%uA", r->connection->number) - p; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; v->data = p; return NGX_OK; } </programlisting> It returns <literal>NGX_ERROR</literal> in case of internal error (for example, failed memory allocation) or <literal>NGX_OK</literal> otherwise. To learn the status of variable evaluation, inspect the flags in <literal>ngx_http_variable_value_t</literal> (see the description <link id="http_existing_variables">above</link>). </para> <para> The <literal>set</literal> handler allows setting the property referenced by the variable. For example, the set handler of the <literal>$limit_rate</literal> variable modifies the request's <literal>limit_rate</literal> field: <programlisting> ... { ngx_string("limit_rate"), ngx_http_variable_request_set_size, ngx_http_variable_request_get_size, offsetof(ngx_http_request_t, limit_rate), NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOCACHEABLE, 0 }, ... static void ngx_http_variable_request_set_size(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { ssize_t s, *sp; ngx_str_t val; val.len = v->len; val.data = v->data; s = ngx_parse_size(&val); if (s == NGX_ERROR) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "invalid size \"%V\"", &val); return; } sp = (ssize_t *) ((char *) r + data); *sp = s; return; } </programlisting> </para> </section> </section> <section name="Complex values" id="http_complex_values"> <para> A complex value, despite its name, provides an easy way to evaluate expressions which can contain text, variables, and their combination. </para> <para> The complex value description in <literal>ngx_http_compile_complex_value</literal> is compiled at the configuration stage into <literal>ngx_http_complex_value_t</literal> which is used at runtime to obtain results of expression evaluation. <programlisting> ngx_str_t *value; ngx_http_complex_value_t cv; ngx_http_compile_complex_value_t ccv; value = cf->args->elts; /* directive arguments */ ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t)); ccv.cf = cf; ccv.value = &value[1]; ccv.complex_value = &cv; ccv.zero = 1; ccv.conf_prefix = 1; if (ngx_http_compile_complex_value(&ccv) != NGX_OK) { return NGX_CONF_ERROR; } </programlisting> Here, <literal>ccv</literal> holds all parameters that are required to initialize the complex value <literal>cv</literal>: <list type="bullet"> <listitem> <literal>cf</literal> — Configuration pointer </listitem> <listitem> <literal>value</literal> — String to be parsed (input) </listitem> <listitem> <literal>complex_value</literal> — Compiled value (output) </listitem> <listitem> <literal>zero</literal> — Flag that enables zero-terminating value </listitem> <listitem> <literal>conf_prefix</literal> — Prefixes the result with the configuration prefix (the directory where nginx is currently looking for configuration) </listitem> <listitem> <literal>root_prefix</literal> — Prefixes the result with the root prefix (the normal nginx installation prefix) </listitem> </list> The <literal>zero</literal> flag is useful when results are to be passed to libraries that require zero-terminated strings, and prefixes are handy when dealing with filenames. </para> <para> Upon successful compilation, <literal>cv.lengths</literal> contains information about the presence of variables in the expression. The NULL value means that the expression contained static text only, and so can be stored in a simple string rather than as a complex value. </para> <para> The <literal>ngx_http_set_complex_value_slot()</literal> is a convenient function used to initialize a complex value completely in the directive declaration itself. </para> <para> At runtime, a complex value can be calculated using the <literal>ngx_http_complex_value()</literal> function: <programlisting> ngx_str_t res; if (ngx_http_complex_value(r, &cv, &res) != NGX_OK) { return NGX_ERROR; } </programlisting> Given the request <literal>r</literal> and previously compiled value <literal>cv</literal>, the function evaluates the expression and writes the result to <literal>res</literal>. </para> </section> <section name="Request redirection" id="http_request_redirection"> <para> An HTTP request is always connected to a location via the <literal>loc_conf</literal> field of the <literal>ngx_http_request_t</literal> structure. This means that at any point the location configuration of any module can be retrieved from the request by calling <literal>ngx_http_get_module_loc_conf(r, module)</literal>. Request location can change several times during the request's lifetime. Initially, a default server location of the default server is assigned to a request. If the request switches to a different server (chosen by the HTTP <header>Host</header> header or SSL SNI extension), the request switches to the default location of that server as well. The next change of the location takes place at the <literal>NGX_HTTP_FIND_CONFIG_PHASE</literal> request phase. At this phase a location is chosen by request URI among all non-named locations configured for the server. The <link doc="../http/ngx_http_rewrite_module.xml">ngx_http_rewrite_module</link> can change the request URI at the <literal>NGX_HTTP_REWRITE_PHASE</literal> request phase as a result of the <link doc="../http/ngx_http_rewrite_module.xml" id="rewrite">rewrite</link> directive and send the request back to the <literal>NGX_HTTP_FIND_CONFIG_PHASE</literal> phase for selection of a new location based on the new URI. </para> <para> It is also possible to redirect a request to a new location at any point by calling one of <literal>ngx_http_internal_redirect(r, uri, args)</literal> or <literal>ngx_http_named_location(r, name)</literal>. </para> <para> The <literal>ngx_http_internal_redirect(r, uri, args)</literal> function changes the request URI and returns the request to the <literal>NGX_HTTP_SERVER_REWRITE_PHASE</literal> phase. The request proceeds with a server default location. Later at <literal>NGX_HTTP_FIND_CONFIG_PHASE</literal> a new location is chosen based on the new request URI. </para> <para> The following example performs an internal redirect with the new request arguments. </para> <programlisting> ngx_int_t ngx_http_foo_redirect(ngx_http_request_t *r) { ngx_str_t uri, args; ngx_str_set(&uri, "/foo"); ngx_str_set(&args, "bar=1"); return ngx_http_internal_redirect(r, &uri, &args); } </programlisting> <para> The function <literal>ngx_http_named_location(r, name)</literal> redirects a request to a named location. The name of the location is passed as the argument. The location is looked up among all named locations of the current server, after which the requests switches to the <literal>NGX_HTTP_REWRITE_PHASE</literal> phase. </para> <para> The following example performs a redirect to a named location @foo. </para> <programlisting> ngx_int_t ngx_http_foo_named_redirect(ngx_http_request_t *r) { ngx_str_t name; ngx_str_set(&name, "foo"); return ngx_http_named_location(r, &name); } </programlisting> <para> Both functions - <literal>ngx_http_internal_redirect(r, uri, args)</literal> and <literal>ngx_http_named_location(r, name)</literal> can be called when nginx modules have already stored some contexts in a request's <literal>ctx</literal> field. It's possible for these contexts to become inconsistent with the new location configuration. To prevent inconsistency, all request contexts are erased by both redirect functions. </para> <para> Calling <literal>ngx_http_internal_redirect(r, uri, args)</literal> or <literal>ngx_http_named_location(r, name)</literal> increases the request <literal>count</literal>. For consistent request reference counting, call <literal>ngx_http_finalize_request(r, NGX_DONE)</literal> after redirecting the request. This will finalize current request code path and decrease the counter. </para> <para> Redirected and rewritten requests become internal and can access the <link doc="../http/ngx_http_core_module.xml" id="internal">internal</link> locations. Internal requests have the <literal>internal</literal> flag set. </para> </section> <section name="Subrequests" id="http_subrequests"> <para> Subrequests are primarily used to insert output of one request into another, possibly mixed with other data. A subrequest looks like a normal request, but shares some data with its parent. In particular, all fields related to client input are shared because a subrequest does not receive any other input from the client. The request field <literal>parent</literal> for a subrequest contains a link to its parent request and is NULL for the main request. The field <literal>main</literal> contains a link to the main request in a group of requests. </para> <para> A subrequest starts in the <literal>NGX_HTTP_SERVER_REWRITE_PHASE</literal> phase. It passes through the same subsequent phases as a normal request and is assigned a location based on its own URI. </para> <para> The output header in a subrequest is always ignored. The <literal>ngx_http_postpone_filter</literal> places the subrequest's output body in the right position relative to other data produced by the parent request. </para> <para> Subrequests are related to the concept of active requests. A request <literal>r</literal> is considered active if <literal>c->data == r</literal>, where <literal>c</literal> is the client connection object. At any given point, only the active request in a request group is allowed to output its buffers to the client. An inactive request can still send its output to the filter chain, but it does not pass beyond the <literal>ngx_http_postpone_filter</literal> and remains buffered by that filter until the request becomes active. Here are some rules of request activation: </para> <list type="bullet"> <listitem> Initially, the main request is active. </listitem> <listitem> The first subrequest of an active request becomes active right after creation. </listitem> <listitem> The <literal>ngx_http_postpone_filter</literal> activates the next request in the active request's subrequest list, once all data prior to that request are sent. </listitem> <listitem> When a request is finalized, its parent is activated. </listitem> </list> <para> Create a subrequest by calling the function <literal>ngx_http_subrequest(r, uri, args, psr, ps, flags)</literal>, where <literal>r</literal> is the parent request, <literal>uri</literal> and <literal>args</literal> are the URI and arguments of the subrequest, <literal>psr</literal> is the output parameter, which receives the newly created subrequest reference, <literal>ps</literal> is a callback object for notifying the parent request that the subrequest is being finalized, and <literal>flags</literal> is bitmask of flags. The following flags are available: </para> <list type="bullet"> <listitem> <literal>NGX_HTTP_SUBREQUEST_IN_MEMORY</literal> - Output is not sent to the client, but rather stored in memory. The flag only affects subrequests which are processed by one of the proxying modules. After a subrequest is finalized its output is available in <literal>r->out</literal> of type <literal>ngx_buf_t</literal>. </listitem> <listitem> <literal>NGX_HTTP_SUBREQUEST_WAITED</literal> - The subrequest's <literal>done</literal> flag is set even if the subrequest is not active when it is finalized. This subrequest flag is used by the SSI filter. </listitem> <listitem> <literal>NGX_HTTP_SUBREQUEST_CLONE</literal> - The subrequest is created as a clone of its parent. It is started at the same location and proceeds from the same phase as the parent request. </listitem> </list> <para> The following example creates a subrequest with the URI of <literal>/foo</literal>. </para> <programlisting> ngx_int_t rc; ngx_str_t uri; ngx_http_request_t *sr; ... ngx_str_set(&uri, "/foo"); rc = ngx_http_subrequest(r, &uri, NULL, &sr, NULL, 0); if (rc == NGX_ERROR) { /* error */ } </programlisting> <para> This example clones the current request and sets a finalization callback for the subrequest. </para> <programlisting> ngx_int_t ngx_http_foo_clone(ngx_http_request_t *r) { ngx_http_request_t *sr; ngx_http_post_subrequest_t *ps; ps = ngx_palloc(r->pool, sizeof(ngx_http_post_subrequest_t)); if (ps == NULL) { return NGX_ERROR; } ps->handler = ngx_http_foo_subrequest_done; ps->data = "foo"; return ngx_http_subrequest(r, &r->uri, &r->args, &sr, ps, NGX_HTTP_SUBREQUEST_CLONE); } ngx_int_t ngx_http_foo_subrequest_done(ngx_http_request_t *r, void *data, ngx_int_t rc) { char *msg = (char *) data; ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, "done subrequest r:%p msg:%s rc:%i", r, msg, rc); return rc; } </programlisting> <para> Subrequests are normally created in a body filter, in which case their output can be treated like the output from any explicit request. This means that eventually the output of a subrequest is sent to the client, after all explicit buffers that are passed before subrequest creation and before any buffers that are passed after creation. This ordering is preserved even for large hierarchies of subrequests. The following example inserts output from a subrequest after all request data buffers, but before the final buffer with the <literal>last_buf</literal> flag. </para> <programlisting> ngx_int_t ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_buf_t *b; ngx_uint_t last; ngx_chain_t *cl, out; ngx_http_request_t *sr; ngx_http_foo_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module); if (ctx == NULL) { return ngx_http_next_body_filter(r, in); } last = 0; for (cl = in; cl; cl = cl->next) { if (cl->buf->last_buf) { cl->buf->last_buf = 0; cl->buf->last_in_chain = 1; cl->buf->sync = 1; last = 1; } } /* Output explicit output buffers */ rc = ngx_http_next_body_filter(r, in); if (rc == NGX_ERROR || !last) { return rc; } /* * Create the subrequest. The output of the subrequest * will automatically be sent after all preceding buffers, * but before the last_buf buffer passed later in this function. */ if (ngx_http_subrequest(r, ctx->uri, NULL, &sr, NULL, 0) != NGX_OK) { return NGX_ERROR; } ngx_http_set_ctx(r, NULL, ngx_http_foo_filter_module); /* Output the final buffer with the last_buf flag */ b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } b->last_buf = 1; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); } </programlisting> <para> A subrequest can also be created for other purposes than data output. For example, the <link doc="../http/ngx_http_auth_request_module.xml"> ngx_http_auth_request_module</link> module creates a subrequest at the <literal>NGX_HTTP_ACCESS_PHASE</literal> phase. To disable output at this point, the <literal>header_only</literal> flag is set on the subrequest. This prevents the subrequest body from being sent to the client. Note that the subrequest's header is never sent to the client. The result of the subrequest can be analyzed in the callback handler. </para> </section> <section name="Request finalization" id="http_request_finalization"> <para> An HTTP request is finalized by calling the function <literal>ngx_http_finalize_request(r, rc)</literal>. It is usually finalized by the content handler after all output buffers are sent to the filter chain. At this point all of the output might not be sent to the client, with some of it remaining buffered somewhere along the filter chain. If it is, the <literal>ngx_http_finalize_request(r, rc)</literal> function automatically installs a special handler <literal>ngx_http_writer(r)</literal> to finish sending the output. A request is also finalized in case of an error or if a standard HTTP response code needs to be returned to the client. </para> <para> The function <literal>ngx_http_finalize_request(r, rc)</literal> expects the following <literal>rc</literal> values: </para> <list type="bullet"> <listitem> <literal>NGX_DONE</literal> - Fast finalization. Decrement the request <literal>count</literal> and destroy the request if it reaches zero. The client connection can be used for more requests after the current request is destroyed. </listitem> <listitem> <literal>NGX_ERROR</literal>, <literal>NGX_HTTP_REQUEST_TIME_OUT</literal> (<literal>408</literal>), <literal>NGX_HTTP_CLIENT_CLOSED_REQUEST</literal> (<literal>499</literal>) - Error finalization. Terminate the request as soon as possible and close the client connection. </listitem> <listitem> <literal>NGX_HTTP_CREATED</literal> (<literal>201</literal>), <literal>NGX_HTTP_NO_CONTENT</literal> (<literal>204</literal>), codes greater than or equal to <literal>NGX_HTTP_SPECIAL_RESPONSE</literal> (<literal>300</literal>) - Special response finalization. For these values nginx either sends to the client a default response page for the code or performs the internal redirect to an <link doc="../http/ngx_http_core_module.xml" id="error_page"/> location if that is configured for the code. </listitem> <listitem> Other codes are considered successful finalization codes and might activate the request writer to finish sending the response body. Once the body is completely sent, the request <literal>count</literal> is decremented. If it reaches zero, the request is destroyed, but the client connection can still be used for other requests. If <literal>count</literal> is positive, there are unfinished activities within the request, which will be finalized at a later point. </listitem> </list> </section> <section name="Request body" id="http_request_body"> <para> For dealing with the body of a client request, nginx provides the <literal>ngx_http_read_client_request_body(r, post_handler)</literal> and <literal>ngx_http_discard_request_body(r)</literal> functions. The first function reads the request body and makes it available via the <literal>request_body</literal> request field. The second function instructs nginx to discard (read and ignore) the request body. One of these functions must be called for every request. Normally, the content handler makes the call. </para> <para> Reading or discarding the client request body from a subrequest is not allowed. It must always be done in the main request. When a subrequest is created, it inherits the parent's <literal>request_body</literal> object which can be used by the subrequest if the main request has previously read the request body. </para> <para> The function <literal>ngx_http_read_client_request_body(r, post_handler)</literal> starts the process of reading the request body. When the body is completely read, the <literal>post_handler</literal> callback is called to continue processing the request. If the request body is missing or has already been read, the callback is called immediately. The function <literal>ngx_http_read_client_request_body(r, post_handler)</literal> allocates the <literal>request_body</literal> request field of type <literal>ngx_http_request_body_t</literal>. The field <literal>bufs</literal> of this object keeps the result as a buffer chain. The body can be saved in memory buffers or file buffers, if the capacity specified by the <link doc="../http/ngx_http_core_module.xml" id="client_body_buffer_size"/> directive is not enough to fit the entire body in memory. </para> <para> The following example reads a client request body and returns its size. </para> <programlisting> ngx_int_t ngx_http_foo_content_handler(ngx_http_request_t *r) { ngx_int_t rc; rc = ngx_http_read_client_request_body(r, ngx_http_foo_init); if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { /* error */ return rc; } return NGX_DONE; } void ngx_http_foo_init(ngx_http_request_t *r) { off_t len; ngx_buf_t *b; ngx_int_t rc; ngx_chain_t *in, out; if (r->request_body == NULL) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } len = 0; for (in = r->request_body->bufs; in; in = in->next) { len += ngx_buf_size(in->buf); } b = ngx_create_temp_buf(r->pool, NGX_OFF_T_LEN); if (b == NULL) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } b->last = ngx_sprintf(b->pos, "%O", len); b->last_buf = (r == r->main) ? 1: 0; b->last_in_chain = 1; r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = b->last - b->pos; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { ngx_http_finalize_request(r, rc); return; } out.buf = b; out.next = NULL; rc = ngx_http_output_filter(r, &out); ngx_http_finalize_request(r, rc); } </programlisting> <para> The following fields of the request determine how the request body is read: </para> <list type="bullet"> <listitem> <literal>request_body_in_single_buf</literal> - Read the body to a single memory buffer. </listitem> <listitem> <literal>request_body_in_file_only</literal> - Always read the body to a file, even if fits in the memory buffer. </listitem> <listitem> <literal>request_body_in_persistent_file</literal> - Do not unlink the file immediately after creation. A file with this flag can be moved to another directory. </listitem> <listitem> <literal>request_body_in_clean_file</literal> - Unlink the file when the request is finalized. This can be useful when a file was supposed to be moved to another directory but was not moved for some reason. </listitem> <listitem> <literal>request_body_file_group_access</literal> - Enable group access to the file by replacing the default 0600 access mask with 0660. </listitem> <listitem> <literal>request_body_file_log_level</literal> - Severity level at which to log file errors. </listitem> <listitem> <literal>request_body_no_buffering</literal> - Read the request body without buffering. </listitem> </list> <para> The <literal>request_body_no_buffering</literal> flag enables the unbuffered mode of reading a request body. In this mode, after calling <literal>ngx_http_read_client_request_body()</literal>, the <literal>bufs</literal> chain might keep only a part of the body. To read the next part, call the <literal>ngx_http_read_unbuffered_request_body(r)</literal> function. The return value <literal>NGX_AGAIN</literal> and the request flag <literal>reading_body</literal> indicate that more data is available. If <literal>bufs</literal> is NULL after calling this function, there is nothing to read at the moment. The request callback <literal>read_event_handler</literal> will be called when the next part of request body is available. </para> </section> <section name="Request body filters" id="http_request_body_filters"> <para> After a request body part is read, it's passed to the request body filter chain by calling the first body filter handler stored in the <literal>ngx_http_top_request_body_filter</literal> variable. It's assumed that every body handler calls the next handler in the chain until the final handler <literal>ngx_http_request_body_save_filter(r, cl)</literal> is called. This handler collects the buffers in <literal>r->request_body->bufs</literal> and writes them to a file if necessary. The last request body buffer has nonzero <literal>last_buf</literal> flag. </para> <para> If a filter is planning to delay data buffers, it should set the flag <literal>r->request_body->filter_need_buffering</literal> to <literal>1</literal> when called for the first time. </para> <para> Following is an example of a simple request body filter that delays request body by one second. </para> <programlisting> #include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> #define NGX_HTTP_DELAY_BODY 1000 typedef struct { ngx_event_t event; ngx_chain_t *out; } ngx_http_delay_body_ctx_t; static ngx_int_t ngx_http_delay_body_filter(ngx_http_request_t *r, ngx_chain_t *in); static void ngx_http_delay_body_cleanup(void *data); static void ngx_http_delay_body_event_handler(ngx_event_t *ev); static ngx_int_t ngx_http_delay_body_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_delay_body_module_ctx = { NULL, /* preconfiguration */ ngx_http_delay_body_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_delay_body_filter_module = { NGX_MODULE_V1, &ngx_http_delay_body_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_request_body_filter_pt ngx_http_next_request_body_filter; static ngx_int_t ngx_http_delay_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_chain_t *cl, *ln; ngx_http_cleanup_t *cln; ngx_http_delay_body_ctx_t *ctx; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "delay request body filter"); ctx = ngx_http_get_module_ctx(r, ngx_http_delay_body_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_delay_body_ctx_t)); if (ctx == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_delay_body_filter_module); r->request_body->filter_need_buffering = 1; } if (ngx_chain_add_copy(r->pool, &ctx->out, in) != NGX_OK) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } if (!ctx->event.timedout) { if (!ctx->event.timer_set) { /* cleanup to remove the timer in case of abnormal termination */ cln = ngx_http_cleanup_add(r, 0); if (cln == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } cln->handler = ngx_http_delay_body_cleanup; cln->data = ctx; /* add timer */ ctx->event.handler = ngx_http_delay_body_event_handler; ctx->event.data = r; ctx->event.log = r->connection->log; ngx_add_timer(&ctx->event, NGX_HTTP_DELAY_BODY); } return ngx_http_next_request_body_filter(r, NULL); } rc = ngx_http_next_request_body_filter(r, ctx->out); for (cl = ctx->out; cl; /* void */) { ln = cl; cl = cl->next; ngx_free_chain(r->pool, ln); } ctx->out = NULL; return rc; } static void ngx_http_delay_body_cleanup(void *data) { ngx_http_delay_body_ctx_t *ctx = data; if (ctx->event.timer_set) { ngx_del_timer(&ctx->event); } } static void ngx_http_delay_body_event_handler(ngx_event_t *ev) { ngx_connection_t *c; ngx_http_request_t *r; r = ev->data; c = r->connection; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "delay request body event"); ngx_post_event(c->read, &ngx_posted_events); } static ngx_int_t ngx_http_delay_body_init(ngx_conf_t *cf) { ngx_http_next_request_body_filter = ngx_http_top_request_body_filter; ngx_http_top_request_body_filter = ngx_http_delay_body_filter; return NGX_OK; } </programlisting> </section> <section name="Response" id="http_response"> <para> In nginx an HTTP response is produced by sending the response header followed by the optional response body. Both header and body are passed through a chain of filters and eventually get written to the client socket. An nginx module can install its handler into the header or body filter chain and process the output coming from the previous handler. </para> <section name="Response header" id="http_response_header"> <para> The <literal>ngx_http_send_header(r)</literal> function sends the output header. Do not call this function until <literal>r->headers_out</literal> contains all of the data required to produce the HTTP response header. The <literal>status</literal> field in <literal>r->headers_out</literal> must always be set. If the response status indicates that a response body follows the header, <literal>content_length_n</literal> can be set as well. The default value for this field is <literal>-1</literal>, which means that the body size is unknown. In this case, chunked transfer encoding is used. To output an arbitrary header, append the <literal>headers</literal> list. </para> <programlisting> static ngx_int_t ngx_http_foo_content_handler(ngx_http_request_t *r) { ngx_int_t rc; ngx_table_elt_t *h; /* send header */ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = 3; /* X-Foo: foo */ h = ngx_list_push(&r->headers_out.headers); if (h == NULL) { return NGX_ERROR; } h->hash = 1; ngx_str_set(&h->key, "X-Foo"); ngx_str_set(&h->value, "foo"); rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; } /* send body */ ... } </programlisting> </section> <section name="Header filters" id="http_header_filters"> <para> The <literal>ngx_http_send_header(r)</literal> function invokes the header filter chain by calling the first header filter handler stored in the <literal>ngx_http_top_header_filter</literal> variable. It's assumed that every header handler calls the next handler in the chain until the final handler <literal>ngx_http_header_filter(r)</literal> is called. The final header handler constructs the HTTP response based on <literal>r->headers_out</literal> and passes it to the <literal>ngx_http_writer_filter</literal> for output. </para> <para> To add a handler to the header filter chain, store its address in the global variable <literal>ngx_http_top_header_filter</literal> at configuration time. The previous handler address is normally stored in a static variable in a module and is called by the newly added handler before exiting. </para> <para> The following example of a header filter module adds the HTTP header "<literal>X-Foo: foo</literal>" to every response with status <literal>200</literal>. </para> <programlisting> #include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r); static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_foo_header_filter_module_ctx = { NULL, /* preconfiguration */ ngx_http_foo_header_filter_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_foo_header_filter_module = { NGX_MODULE_V1, &ngx_http_foo_header_filter_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_output_header_filter_pt ngx_http_next_header_filter; static ngx_int_t ngx_http_foo_header_filter(ngx_http_request_t *r) { ngx_table_elt_t *h; /* * The filter handler adds "X-Foo: foo" header * to every HTTP 200 response */ if (r->headers_out.status != NGX_HTTP_OK) { return ngx_http_next_header_filter(r); } h = ngx_list_push(&r->headers_out.headers); if (h == NULL) { return NGX_ERROR; } h->hash = 1; ngx_str_set(&h->key, "X-Foo"); ngx_str_set(&h->value, "foo"); return ngx_http_next_header_filter(r); } static ngx_int_t ngx_http_foo_header_filter_init(ngx_conf_t *cf) { ngx_http_next_header_filter = ngx_http_top_header_filter; ngx_http_top_header_filter = ngx_http_foo_header_filter; return NGX_OK; } </programlisting> </section> </section> <section name="Response body" id="http_response_body"> <para> To send the response body, call the <literal>ngx_http_output_filter(r, cl)</literal> function. The function can be called multiple times. Each time, it sends a part of the response body in the form of a buffer chain. Set the <literal>last_buf</literal> flag in the last body buffer. </para> <para> The following example produces a complete HTTP response with "foo" as its body. For the example to work as subrequest as well as a main request, the <literal>last_in_chain</literal> flag is set in the last buffer of the output. The <literal>last_buf</literal> flag is set only for the main request because the last buffer for a subrequest does not end the entire output. </para> <programlisting> static ngx_int_t ngx_http_bar_content_handler(ngx_http_request_t *r) { ngx_int_t rc; ngx_buf_t *b; ngx_chain_t out; /* send header */ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = 3; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; } /* send body */ b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } b->last_buf = (r == r->main) ? 1: 0; b->last_in_chain = 1; b->memory = 1; b->pos = (u_char *) "foo"; b->last = b->pos + 3; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); } </programlisting> </section> <section name="Response body filters" id="http_response_body_filters"> <para> The function <literal>ngx_http_output_filter(r, cl)</literal> invokes the body filter chain by calling the first body filter handler stored in the <literal>ngx_http_top_body_filter</literal> variable. It's assumed that every body handler calls the next handler in the chain until the final handler <literal>ngx_http_write_filter(r, cl)</literal> is called. </para> <para> A body filter handler receives a chain of buffers. The handler is supposed to process the buffers and pass a possibly new chain to the next handler. It's worth noting that the chain links <literal>ngx_chain_t</literal> of the incoming chain belong to the caller, and must not be reused or changed. Right after the handler completes, the caller can use its output chain links to keep track of the buffers it has sent. To save the buffer chain or to substitute some buffers before passing to the next filter, a handler needs to allocate its own chain links. </para> <para> Following is an example of a simple body filter that counts the number of bytes in the body. The result is available as the <literal>$counter</literal> variable which can be used in the access log. </para> <programlisting> #include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> typedef struct { off_t count; } ngx_http_counter_filter_ctx_t; static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in); static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf); static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf); static ngx_http_module_t ngx_http_counter_filter_module_ctx = { ngx_http_counter_add_variables, /* preconfiguration */ ngx_http_counter_filter_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; ngx_module_t ngx_http_counter_filter_module = { NGX_MODULE_V1, &ngx_http_counter_filter_module_ctx, /* module context */ NULL, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_output_body_filter_pt ngx_http_next_body_filter; static ngx_str_t ngx_http_counter_name = ngx_string("counter"); static ngx_int_t ngx_http_counter_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_chain_t *cl; ngx_http_counter_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_counter_filter_ctx_t)); if (ctx == NULL) { return NGX_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_counter_filter_module); } for (cl = in; cl; cl = cl->next) { ctx->count += ngx_buf_size(cl->buf); } return ngx_http_next_body_filter(r, in); } static ngx_int_t ngx_http_counter_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { u_char *p; ngx_http_counter_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_counter_filter_module); if (ctx == NULL) { v->not_found = 1; return NGX_OK; } p = ngx_pnalloc(r->pool, NGX_OFF_T_LEN); if (p == NULL) { return NGX_ERROR; } v->data = p; v->len = ngx_sprintf(p, "%O", ctx->count) - p; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; return NGX_OK; } static ngx_int_t ngx_http_counter_add_variables(ngx_conf_t *cf) { ngx_http_variable_t *var; var = ngx_http_add_variable(cf, &ngx_http_counter_name, 0); if (var == NULL) { return NGX_ERROR; } var->get_handler = ngx_http_counter_variable; return NGX_OK; } static ngx_int_t ngx_http_counter_filter_init(ngx_conf_t *cf) { ngx_http_next_body_filter = ngx_http_top_body_filter; ngx_http_top_body_filter = ngx_http_counter_body_filter; return NGX_OK; } </programlisting> </section> <section name="Building filter modules" id="http_building_filter_modules"> <para> When writing a body or header filter, pay special attention to the filter's position in the filter order. There's a number of header and body filters registered by nginx standard modules. The nginx standard modules register a number of head and body filters and it's important to register a new filter module in the right place with respect to them. Normally, modules register filters in their postconfiguration handlers. The order in which filters are called during processing is obviously the reverse of the order in which they are registered. </para> <para> For third-party filter modules nginx provides a special slot <literal>HTTP_AUX_FILTER_MODULES</literal>. To register a filter module in this slot, set the <literal>ngx_module_type</literal> variable to <literal>HTTP_AUX_FILTER</literal> in the module's configuration. </para> <para> The following example shows a filter module config file assuming for a module with just one source file, <literal>ngx_http_foo_filter_module.c</literal>. </para> <programlisting> ngx_module_type=HTTP_AUX_FILTER ngx_module_name=ngx_http_foo_filter_module ngx_module_srcs="$ngx_addon_dir/ngx_http_foo_filter_module.c" . auto/module </programlisting> </section> <section name="Buffer reuse" id="http_body_buffers_reuse"> <para> When issuing or altering a stream of buffers, it's often desirable to reuse the allocated buffers. A standard and widely adopted approach in nginx code is to keep two buffer chains for this purpose: <literal>free</literal> and <literal>busy</literal>. The <literal>free</literal> chain keeps all free buffers, which can be reused. The <literal>busy</literal> chain keeps all buffers sent by the current module that are still in use by some other filter handler. A buffer is considered in use if its size is greater than zero. Normally, when a buffer is consumed by a filter, its <literal>pos</literal> (or <literal>file_pos</literal> for a file buffer) is moved towards <literal>last</literal> (<literal>file_last</literal> for a file buffer). Once a buffer is completely consumed, it's ready to be reused. To add newly freed buffers to the <literal>free</literal> chain it's enough to iterate over the <literal>busy</literal> chain and move the zero size buffers at the head of it to <literal>free</literal>. This operation is so common that there is a special function for it, <literal>ngx_chain_update_chains(free, busy, out, tag)</literal>. The function appends the output chain <literal>out</literal> to <literal>busy</literal> and moves free buffers from the top of <literal>busy</literal> to <literal>free</literal>. Only the buffers with the specified <literal>tag</literal> are reused. This lets a module reuse only the buffers that it allocated itself. </para> <para> The following example is a body filter that inserts the string “foo” before each incoming buffer. The new buffers allocated by the module are reused if possible. Note that for this example to work properly, setting up a <link id="http_header_filters">header filter</link> and resetting <literal>content_length_n</literal> to <literal>-1</literal> is also required, but the relevant code is not provided here. </para> <programlisting> typedef struct { ngx_chain_t *free; ngx_chain_t *busy; } ngx_http_foo_filter_ctx_t; ngx_int_t ngx_http_foo_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { ngx_int_t rc; ngx_buf_t *b; ngx_chain_t *cl, *tl, *out, **ll; ngx_http_foo_filter_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_foo_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_foo_filter_ctx_t)); if (ctx == NULL) { return NGX_ERROR; } ngx_http_set_ctx(r, ctx, ngx_http_foo_filter_module); } /* create a new chain "out" from "in" with all the changes */ ll = &out; for (cl = in; cl; cl = cl->next) { /* append "foo" in a reused buffer if possible */ tl = ngx_chain_get_free_buf(r->pool, &ctx->free); if (tl == NULL) { return NGX_ERROR; } b = tl->buf; b->tag = (ngx_buf_tag_t) &ngx_http_foo_filter_module; b->memory = 1; b->pos = (u_char *) "foo"; b->last = b->pos + 3; *ll = tl; ll = &tl->next; /* append the next incoming buffer */ tl = ngx_alloc_chain_link(r->pool); if (tl == NULL) { return NGX_ERROR; } tl->buf = cl->buf; *ll = tl; ll = &tl->next; } *ll = NULL; /* send the new chain */ rc = ngx_http_next_body_filter(r, out); /* update "busy" and "free" chains for reuse */ ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &out, (ngx_buf_tag_t) &ngx_http_foo_filter_module); return rc; } </programlisting> </section> <section name="Load balancing" id="http_load_balancing"> <para> The <link doc="../http/ngx_http_upstream_module.xml">ngx_http_upstream_module</link> provides the basic functionality needed to pass requests to remote servers. Modules that implement specific protocols, such as HTTP or FastCGI, use this functionality. The module also provides an interface for creating custom load-balancing modules and implements a default round-robin method. </para> <para> The <link doc="../http/ngx_http_upstream_module.xml" id="least_conn"/> and <link doc="../http/ngx_http_upstream_module.xml" id="hash"/> modules implement alternative load-balancing methods, but are actually implemented as extensions of the upstream round-robin module and share a lot of code with it, such as the representation of a server group. The <link doc="../http/ngx_http_upstream_module.xml" id="keepalive"/> module is an independent module that extends upstream functionality. </para> <para> The <link doc="../http/ngx_http_upstream_module.xml">ngx_http_upstream_module</link> can be configured explicitly by placing the corresponding <link doc="../http/ngx_http_upstream_module.xml" id="upstream"/> block into the configuration file, or implicitly by using directives such as <link doc="../http/ngx_http_proxy_module.xml" id="proxy_pass"/> that accept a URL that gets evaluated at some point into a list of servers. The alternative load-balancing methods are available only with an explicit upstream configuration. The upstream module configuration has its own directive context <literal>NGX_HTTP_UPS_CONF</literal>. The structure is defined as follows: <programlisting> struct ngx_http_upstream_srv_conf_s { ngx_http_upstream_peer_t peer; void **srv_conf; ngx_array_t *servers; /* ngx_http_upstream_server_t */ ngx_uint_t flags; ngx_str_t host; u_char *file_name; ngx_uint_t line; in_port_t port; ngx_uint_t no_port; /* unsigned no_port:1 */ #if (NGX_HTTP_UPSTREAM_ZONE) ngx_shm_zone_t *shm_zone; #endif }; </programlisting> <list type="bullet"> <listitem> <literal>srv_conf</literal> — Configuration context of upstream modules. </listitem> <listitem> <literal>servers</literal> — Array of <literal>ngx_http_upstream_server_t</literal>, the result of parsing a set of <link doc="../http/ngx_http_upstream_module.xml" id="server"/> directives in the <literal>upstream</literal> block. </listitem> <listitem> <literal>flags</literal> — Flags that mostly mark which features are supported by the load-balancing method. The features are configured as parameters of the <link doc="../http/ngx_http_upstream_module.xml" id="server"/> directive: <list type="bullet"> <listitem> <literal>NGX_HTTP_UPSTREAM_CREATE</literal> — Distinguishes explicitly defined upstreams from those that are automatically created by the <link doc="../http/ngx_http_proxy_module.xml" id="proxy_pass"/> directive and “friends” (FastCGI, SCGI, etc.) </listitem> <listitem> <literal>NGX_HTTP_UPSTREAM_WEIGHT</literal> — The “<literal>weight</literal>” parameter is supported </listitem> <listitem> <literal>NGX_HTTP_UPSTREAM_MAX_FAILS</literal> — The “<literal>max_fails</literal>” parameter is supported </listitem> <listitem> <literal>NGX_HTTP_UPSTREAM_FAIL_TIMEOUT</literal> — The “<literal>fail_timeout</literal>” parameter is supported </listitem> <listitem> <literal>NGX_HTTP_UPSTREAM_DOWN</literal> — The “<literal>down</literal>” parameter is supported </listitem> <listitem> <literal>NGX_HTTP_UPSTREAM_BACKUP</literal> — The “<literal>backup</literal>” parameter is supported </listitem> <listitem> <literal>NGX_HTTP_UPSTREAM_MAX_CONNS</literal> — The “<literal>max_conns</literal>” parameter is supported </listitem> </list> </listitem> <listitem> <literal>host</literal> — Name of the upstream. </listitem> <listitem> <literal>file_name, line</literal> — Name of the configuration file and the line where the <literal>upstream</literal> block is located. </listitem> <listitem> <literal>port</literal> and <literal>no_port</literal> — Not used for explicitly defined upstream groups. </listitem> <listitem> <literal>shm_zone</literal> — Shared memory zone used by this upstream group, if any. </listitem> <listitem> <literal>peer</literal> — object that holds generic methods for initializing upstream configuration: <programlisting> typedef struct { ngx_http_upstream_init_pt init_upstream; ngx_http_upstream_init_peer_pt init; void *data; } ngx_http_upstream_peer_t; </programlisting> A module that implements a load-balancing algorithm must set these methods and initialize private <literal>data</literal>. If <literal>init_upstream</literal> was not initialized during configuration parsing, <literal>ngx_http_upstream_module</literal> sets it to the default <literal>ngx_http_upstream_init_round_robin</literal> algorithm. <list type="bullet"> <listitem> <literal>init_upstream(cf, us)</literal> — Configuration-time method responsible for initializing a group of servers and initializing the <literal>init()</literal> method in case of success. A typical load-balancing module uses a list of servers in the <literal>upstream</literal> block to create an efficient data structure that it uses and saves its own configuration to the <literal>data</literal> field. </listitem> <listitem> <literal>init(r, us)</literal> — Initializes a per-request <literal>ngx_http_upstream_peer_t.peer</literal> structure that is used for load balancing (not to be confused with the <literal>ngx_http_upstream_srv_conf_t.peer</literal> described above which is per-upstream). It is passed as the <literal>data</literal> argument to all callbacks that deal with server selection. </listitem> </list> </listitem> </list> </para> <para> When nginx has to pass a request to another host for processing, it uses the configured load-balancing method to obtain an address to connect to. The method is obtained from the <literal>ngx_http_upstream_t.peer</literal> object of type <literal>ngx_peer_connection_t</literal>: <programlisting> struct ngx_peer_connection_s { ... struct sockaddr *sockaddr; socklen_t socklen; ngx_str_t *name; ngx_uint_t tries; ngx_event_get_peer_pt get; ngx_event_free_peer_pt free; ngx_event_notify_peer_pt notify; void *data; #if (NGX_SSL || NGX_COMPAT) ngx_event_set_peer_session_pt set_session; ngx_event_save_peer_session_pt save_session; #endif ... }; </programlisting> The structure has the following fields: <list type="bullet"> <listitem> <literal>sockaddr</literal>, <literal>socklen</literal>, <literal>name</literal> — Address of the upstream server to connect to; this is the output parameter of a load-balancing method. </listitem> <listitem> <literal>data</literal> — The per-request data of a load-balancing method; keeps the state of the selection algorithm and usually includes the link to the upstream configuration. It is passed as an argument to all methods that deal with server selection (see <link id="lb_method_get">below</link>). </listitem> <listitem> <literal>tries</literal> — Allowed <link doc="../http/ngx_http_proxy_module.xml" id="proxy_next_upstream_tries">number</link> of attempts to connect to an upstream server. </listitem> <listitem> <literal>get</literal>, <literal>free</literal>, <literal>notify</literal>, <literal>set_session</literal>, and <literal>save_session</literal> - Methods of the load-balancing module, described below. </listitem> </list> </para> <para> All methods accept at least two arguments: a peer connection object <literal>pc</literal> and the <literal>data</literal> created by <literal>ngx_http_upstream_srv_conf_t.peer.init()</literal>. Note that it might differ from <literal>pc.data</literal> due to “chaining” of load-balancing modules. </para> <para> <list type="bullet"> <listitem id="lb_method_get"> <literal>get(pc, data)</literal> — The method called when the upstream module is ready to pass a request to an upstream server and needs to know its address. The method has to fill the <literal>sockaddr</literal>, <literal>socklen</literal>, and <literal>name</literal> fields of <literal>ngx_peer_connection_t</literal> structure. The return is one of: <list type="bullet"> <listitem> <literal>NGX_OK</literal> — Server was selected. </listitem> <listitem> <literal>NGX_ERROR</literal> — Internal error occurred. </listitem> <listitem> <literal>NGX_BUSY</literal> — no servers are currently available. This can happen due to many reasons, including: the dynamic server group is empty, all servers in the group are in the failed state, or all servers in the group are already handling the maximum number of connections. </listitem> <listitem> <literal>NGX_DONE</literal> — the underlying connection was reused and there is no need to create a new connection to the upstream server. This value is set by the <literal>keepalive</literal> module. </listitem> <!-- <listitem> <literal>NGX_ABORT</literal> — the request was queued and the further processing of this request should be postponed. This value is set by the <literal>queue</literal> module. </listitem> --> </list> </listitem> <listitem> <literal>free(pc, data, state)</literal> — The method called when an upstream module has finished work with a particular server. The <literal>state</literal> argument is the completion status of the upstream connection, a bitmask with the following possible values: <list type="bullet"> <listitem> <literal>NGX_PEER_FAILED</literal> — Attempt was <link doc="../http/ngx_http_upstream_module.xml" id="max_fails">unsuccessful</link> </listitem> <listitem> <literal>NGX_PEER_NEXT</literal> — A special case when upstream server returns codes <literal>403</literal> or <literal>404</literal>, which are not considered a <link doc="../http/ngx_http_upstream_module.xml" id="max_fails">failure</link>. </listitem> <listitem> <literal>NGX_PEER_KEEPALIVE</literal> — Currently unused </listitem> </list> This method also decrements the <literal>tries</literal> counter. </listitem> <listitem> <literal>notify(pc, data, type)</literal> — Currently unused in the OSS version. </listitem> <listitem> <literal>set_session(pc, data)</literal> and <literal>save_session(pc, data)</literal> — SSL-specific methods that enable caching sessions to upstream servers. The implementation is provided by the round-robin balancing method. </listitem> </list> </para> </section> </section> <section name="Examples" id="examples"> <para> The <link url="http://hg.nginx.org/nginx-dev-examples">nginx-dev-examples</link> repository provides nginx module examples. </para> </section> <section name="Code style" id="code_style"> <section name="General rules" id="code_style_general_rules"> <para> <list type="bullet"> <listitem> maximum text width is 80 characters </listitem> <listitem> indentation is 4 spaces </listitem> <listitem> no tabs, no trailing spaces </listitem> <listitem> list elements on the same line are separated with spaces </listitem> <listitem> hexadecimal literals are lowercase </listitem> <listitem> file names, function and type names, and global variables have the <literal>ngx_</literal> or more specific prefix such as <literal>ngx_http_</literal> and <literal>ngx_mail_</literal> </listitem> </list> <programlisting> size_t ngx_utf8_length(u_char *p, size_t n) { u_char c, *last; size_t len; last = p + n; for (len = 0; p < last; len++) { c = *p; if (c < 0x80) { p++; continue; } if (ngx_utf8_decode(&p, last - p) > 0x10ffff) { /* invalid UTF-8 */ return n; } } return len; } </programlisting> </para> </section> <section name="Files" id="code_style_files"> <para> A typical source file may contain the following sections separated by two empty lines: <list type="bullet"> <listitem> copyright statements </listitem> <listitem> includes </listitem> <listitem> preprocessor definitions </listitem> <listitem> type definitions </listitem> <listitem> function prototypes </listitem> <listitem> variable definitions </listitem> <listitem> function definitions </listitem> </list> </para> <para> Copyright statements look like this: <programlisting> /* * Copyright (C) Author Name * Copyright (C) Organization, Inc. */ </programlisting> If the file is modified significantly, the list of authors should be updated, the new author is added to the top. </para> <para> The <literal>ngx_config.h</literal> and <literal>ngx_core.h</literal> files are always included first, followed by one of <literal>ngx_http.h</literal>, <literal>ngx_stream.h</literal>, or <literal>ngx_mail.h</literal>. Then follow optional external header files: <programlisting> #include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> #include <libxml/parser.h> #include <libxml/tree.h> #include <libxslt/xslt.h> #if (NGX_HAVE_EXSLT) #include <libexslt/exslt.h> #endif </programlisting> </para> <para> Header files should include the so called "header protection": <programlisting> #ifndef _NGX_PROCESS_CYCLE_H_INCLUDED_ #define _NGX_PROCESS_CYCLE_H_INCLUDED_ ... #endif /* _NGX_PROCESS_CYCLE_H_INCLUDED_ */ </programlisting> </para> </section> <section name="Comments" id="code_style_comments"> <para> <list type="bullet"> <listitem> “<literal>//</literal>” comments are not used </listitem> <listitem> text is written in English, American spelling is preferred </listitem> <listitem> multi-line comments are formatted like this: <programlisting> /* * The red-black tree code is based on the algorithm described in * the "Introduction to Algorithms" by Cormen, Leiserson and Rivest. */ </programlisting> <programlisting> /* find the server configuration for the address:port */ </programlisting> </listitem> </list> </para> </section> <section name="Preprocessor" id="code_style_preprocessor"> <para> Macro names start from <literal>ngx_</literal> or <literal>NGX_</literal> (or more specific) prefix. Macro names for constants are uppercase. Parameterized macros and macros for initializers are lowercase. The macro name and value are separated by at least two spaces: <programlisting> #define NGX_CONF_BUFFER 4096 #define ngx_buf_in_memory(b) (b->temporary || b->memory || b->mmap) #define ngx_buf_size(b) \ (ngx_buf_in_memory(b) ? (off_t) (b->last - b->pos): \ (b->file_last - b->file_pos)) #define ngx_null_string { 0, NULL } </programlisting> Conditions are inside parentheses, negation is outside: <programlisting> #if (NGX_HAVE_KQUEUE) ... #elif ((NGX_HAVE_DEVPOLL && !(NGX_TEST_BUILD_DEVPOLL)) \ || (NGX_HAVE_EVENTPORT && !(NGX_TEST_BUILD_EVENTPORT))) ... #elif (NGX_HAVE_EPOLL && !(NGX_TEST_BUILD_EPOLL)) ... #elif (NGX_HAVE_POLL) ... #else /* select */ ... #endif /* NGX_HAVE_KQUEUE */ </programlisting> </para> </section> <section name="Types" id="code_style_types"> <para> Type names end with the “<literal>_t</literal>” suffix. A defined type name is separated by at least two spaces: <programlisting> typedef ngx_uint_t ngx_rbtree_key_t; </programlisting> </para> <para> Structure types are defined using <literal>typedef</literal>. Inside structures, member types and names are aligned: <programlisting> typedef struct { size_t len; u_char *data; } ngx_str_t; </programlisting> Keep alignment identical among different structures in the file. A structure that points to itself has the name, ending with “<literal>_s</literal>”. Adjacent structure definitions are separated with two empty lines: <programlisting> typedef struct ngx_list_part_s ngx_list_part_t; struct ngx_list_part_s { void *elts; ngx_uint_t nelts; ngx_list_part_t *next; }; typedef struct { ngx_list_part_t *last; ngx_list_part_t part; size_t size; ngx_uint_t nalloc; ngx_pool_t *pool; } ngx_list_t; </programlisting> Each structure member is declared on its own line: <programlisting> typedef struct { ngx_uint_t hash; ngx_str_t key; ngx_str_t value; u_char *lowcase_key; } ngx_table_elt_t; </programlisting> </para> <para> Function pointers inside structures have defined types ending with “<literal>_pt</literal>”: <programlisting> typedef ssize_t (*ngx_recv_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); typedef ssize_t (*ngx_send_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ngx_chain_t *(*ngx_send_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); typedef struct { ngx_recv_pt recv; ngx_recv_chain_pt recv_chain; ngx_recv_pt udp_recv; ngx_send_pt send; ngx_send_pt udp_send; ngx_send_chain_pt udp_send_chain; ngx_send_chain_pt send_chain; ngx_uint_t flags; } ngx_os_io_t; </programlisting> </para> <para> Enumerations have types ending with “<literal>_e</literal>”: <programlisting> typedef enum { ngx_http_fastcgi_st_version = 0, ngx_http_fastcgi_st_type, ... ngx_http_fastcgi_st_padding } ngx_http_fastcgi_state_e; </programlisting> </para> </section> <section name="Variables" id="code_style_variables"> <para> Variables are declared sorted by length of a base type, then alphabetically. Type names and variable names are aligned. The type and name “columns” are separated with two spaces. Large arrays are put at the end of a declaration block: <programlisting> u_char | | *rv, *p; ngx_conf_t | | *cf; ngx_uint_t | | i, j, k; unsigned int | | len; struct sockaddr | | *sa; const unsigned char | | *data; ngx_peer_connection_t | | *pc; ngx_http_core_srv_conf_t | |**cscfp; ngx_http_upstream_srv_conf_t| | *us, *uscf; u_char | | text[NGX_SOCKADDR_STRLEN]; </programlisting> </para> <para> Static and global variables may be initialized on declaration: <programlisting> static ngx_str_t ngx_http_memcached_key = ngx_string("memcached_key"); </programlisting> <programlisting> static ngx_uint_t mday[] = { 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 }; </programlisting> <programlisting> static uint32_t ngx_crc32_table16[] = { 0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac, ... 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c }; </programlisting> </para> <para> There is a bunch of commonly used type/name combinations: <programlisting> u_char *rv; ngx_int_t rc; ngx_conf_t *cf; ngx_connection_t *c; ngx_http_request_t *r; ngx_peer_connection_t *pc; ngx_http_upstream_srv_conf_t *us, *uscf; </programlisting> </para> </section> <section name="Functions" id="code_style_functions"> <para> All functions (even static ones) should have prototypes. Prototypes include argument names. Long prototypes are wrapped with a single indentation on continuation lines: <programlisting> static char *ngx_http_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static ngx_int_t ngx_http_init_phases(ngx_conf_t *cf, ngx_http_core_main_conf_t *cmcf); static char *ngx_http_merge_servers(ngx_conf_t *cf, ngx_http_core_main_conf_t *cmcf, ngx_http_module_t *module, ngx_uint_t ctx_index); </programlisting> The function name in a definition starts with a new line. The function body opening and closing braces are on separate lines. The body of a function is indented. There are two empty lines between functions: <programlisting> static ngx_int_t ngx_http_find_virtual_server(ngx_http_request_t *r, u_char *host, size_t len) { ... } static ngx_int_t ngx_http_add_addresses(ngx_conf_t *cf, ngx_http_core_srv_conf_t *cscf, ngx_http_conf_port_t *port, ngx_http_listen_opt_t *lsopt) { ... } </programlisting> There is no space after the function name and opening parenthesis. Long function calls are wrapped such that continuation lines start from the position of the first function argument. If this is impossible, format the first continuation line such that it ends at position 79: <programlisting> ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http header: \"%V: %V\"", &h->key, &h->value); hc->busy = ngx_palloc(r->connection->pool, cscf->large_client_header_buffers.num * sizeof(ngx_buf_t *)); </programlisting> The <literal>ngx_inline</literal> macro should be used instead of <literal>inline</literal>: <programlisting> static ngx_inline void ngx_cpuid(uint32_t i, uint32_t *buf); </programlisting> </para> </section> <section name="Expressions" id="code_style_expressions"> <para> Binary operators except “<literal>.</literal>” and “<literal>−></literal>” should be separated from their operands by one space. Unary operators and subscripts are not separated from their operands by spaces: <programlisting> width = width * 10 + (*fmt++ - '0'); </programlisting> <programlisting> ch = (u_char) ((decoded << 4) + (ch - '0')); </programlisting> <programlisting> r->exten.data = &r->uri.data[i + 1]; </programlisting> </para> <para> Type casts are separated by one space from casted expressions. An asterisk inside type cast is separated with space from type name: <programlisting> len = ngx_sock_ntop((struct sockaddr *) sin6, p, len, 1); </programlisting> </para> <para> If an expression does not fit into single line, it is wrapped. The preferred point to break a line is a binary operator. The continuation line is lined up with the start of expression: <programlisting> if (status == NGX_HTTP_MOVED_PERMANENTLY || status == NGX_HTTP_MOVED_TEMPORARILY || status == NGX_HTTP_SEE_OTHER || status == NGX_HTTP_TEMPORARY_REDIRECT || status == NGX_HTTP_PERMANENT_REDIRECT) { ... } </programlisting> <programlisting> p->temp_file->warn = "an upstream response is buffered " "to a temporary file"; </programlisting> As a last resort, it is possible to wrap an expression so that the continuation line ends at position 79: <programlisting> hinit->hash = ngx_pcalloc(hinit->pool, sizeof(ngx_hash_wildcard_t) + size * sizeof(ngx_hash_elt_t *)); </programlisting> The above rules also apply to sub-expressions, where each sub-expression has its own indentation level: <programlisting> if (((u->conf->cache_use_stale & NGX_HTTP_UPSTREAM_FT_UPDATING) || c->stale_updating) && !r->background && u->conf->cache_background_update) { ... } </programlisting> Sometimes, it is convenient to wrap an expression after a cast. In this case, the continuation line is indented: <programlisting> node = (ngx_rbtree_node_t *) ((u_char *) lr - offsetof(ngx_rbtree_node_t, color)); </programlisting> </para> <para> Pointers are explicitly compared to <literal>NULL</literal> (not <literal>0</literal>): <programlisting> if (ptr != NULL) { ... } </programlisting> </para> </section> <section name="Conditionals and Loops" id="code_style_conditionals_and_loops"> <para> The “<literal>if</literal>” keyword is separated from the condition by one space. Opening brace is located on the same line, or on a dedicated line if the condition takes several lines. Closing brace is located on a dedicated line, optionally followed by “<literal>else if</literal> / <literal>else</literal>”. Usually, there is an empty line before the “<literal>else if</literal> / <literal>else</literal>” part: <programlisting> if (node->left == sentinel) { temp = node->right; subst = node; } else if (node->right == sentinel) { temp = node->left; subst = node; } else { subst = ngx_rbtree_min(node->right, sentinel); if (subst->left != sentinel) { temp = subst->left; } else { temp = subst->right; } } </programlisting> </para> <para> Similar formatting rules are applied to “<literal>do</literal>” and “<literal>while</literal>” loops: <programlisting> while (p < last && *p == ' ') { p++; } </programlisting> <programlisting> do { ctx->node = rn; ctx = ctx->next; } while (ctx); </programlisting> </para> <para> The “<literal>switch</literal>” keyword is separated from the condition by one space. Opening brace is located on the same line. Closing brace is located on a dedicated line. The “<literal>case</literal>” keywords are lined up with “<literal>switch</literal>”: <programlisting> switch (ch) { case '!': looked = 2; state = ssi_comment0_state; break; case '<': copy_end = p; break; default: copy_end = p; looked = 0; state = ssi_start_state; break; } </programlisting> </para> <para> Most “<literal>for</literal>” loops are formatted like this: <programlisting> for (i = 0; i < ccf->env.nelts; i++) { ... } </programlisting> <programlisting> for (q = ngx_queue_head(locations); q != ngx_queue_sentinel(locations); q = ngx_queue_next(q)) { ... } </programlisting> If some part of the “<literal>for</literal>” statement is omitted, this is indicated by the “<literal>/* void */</literal>” comment: <programlisting> for (i = 0; /* void */ ; i++) { ... } </programlisting> A loop with an empty body is also indicated by the “<literal>/* void */</literal>” comment which may be put on the same line: <programlisting> for (cl = *busy; cl->next; cl = cl->next) { /* void */ } </programlisting> An endless loop looks like this: <programlisting> for ( ;; ) { ... } </programlisting> </para> </section> <section name="Labels" id="code_style_labels"> <para> Labels are surrounded with empty lines and are indented at the previous level: <programlisting> if (i == 0) { u->err = "host not found"; goto failed; } u->addrs = ngx_pcalloc(pool, i * sizeof(ngx_addr_t)); if (u->addrs == NULL) { goto failed; } u->naddrs = i; ... return NGX_OK; failed: freeaddrinfo(res); return NGX_ERROR; </programlisting> </para> </section> </section> <section name="Debugging memory issues" id="debug_memory"> <para> To debug memory issues such as buffer overruns or use-after-free errors, you can use the <link url="https://en.wikipedia.org/wiki/AddressSanitizer"> AddressSanitizer</link> (ASan) supported by some modern compilers. To enable ASan with <literal>gcc</literal> and <literal>clang</literal>, use the <literal>-fsanitize=address</literal> compiler and linker option. When building nginx, this can be done by adding the option to <literal>--with-cc-opt</literal> and <literal>--with-ld-opt</literal> parameters of the <literal>configure</literal> script. </para> <para> Since most allocations in nginx are made from nginx internal <link id="pool">pool</link>, enabling ASan may not always be enough to debug memory issues. The internal pool allocates a big chunk of memory from the system and cuts smaller allocations from it. However, this mechanism can be disabled by setting the <literal>NGX_DEBUG_PALLOC</literal> macro to <literal>1</literal>. In this case, allocations are passed directly to the system allocator giving it full control over the buffers boundaries. </para> <para> The following configuration line summarizes the information provided above. It is recommended while developing third-party modules and testing nginx on different platforms. </para> <programlisting> auto/configure --with-cc-opt='-fsanitize=address -DNGX_DEBUG_PALLOC=1' --with-ld-opt=-fsanitize=address </programlisting> </section> <section name="Common Pitfalls" id="common_pitfals"> <section name="Writing a C module" id="module_pitfall"> <para> The most common pitfall is an attempt to write a full-fledged C module when it can be avoided. In most cases your task can be accomplished by creating a proper configuration. If writing a module is inevitable, try to make it as small and simple as possible. For example, a module can only export some <link id="http_variables">variables</link>. </para> <para> Before starting a module, consider the following questions: <list type="bullet"> <listitem> Is it possible to implement a desired feature using already <link doc="../../docs/index.xml">available modules</link>? </listitem> <listitem> Is it possible to solve an issue using built-in scripting languages, such as <link doc="../http/ngx_http_perl_module.xml">Perl</link> or <link doc="../njs/index.xml">njs</link>? </listitem> </list> </para> </section> <section name="C Strings" id="c_strings"> <para> The most used string type in nginx, <link id="string_overview">ngx_str_t</link> is not a C-Style zero-terminated string. You cannot pass the data to standard C library functions such as <c-func>strlen</c-func> or <c-func>strstr</c-func>. Instead, nginx <link id="string_overview">counterparts</link> that accept either <literal>ngx_str_t</literal> should be used or pointer to data and a length. However, there is a case when <literal>ngx_str_t</literal> holds a pointer to a zero-terminated string: strings that come as a result of configuration file parsing are zero-terminated. </para> </section> <section name="Global Variables" id="global_variables"> <para> Avoid using global variables in your modules. Most likely this is an error to have a global variable. Any global data should be tied to a <link id="cycle">configuration cycle</link> and be allocated from the corresponding <link id="pool">memory pool</link>. This allows nginx to perform graceful configuration reloads. An attempt to use global variables will likely break this feature, because it will be impossible to have two configurations at the same time and get rid of them. Sometimes global variables are required. In this case, special attention is needed to manage reconfiguration properly. Also, check if libraries used by your code have implicit global state that may be broken on reload. </para> </section> <section name="Manual Memory Management" id="manual_memory_management"> <para> Instead of dealing with malloc/free approach which is error prone, learn how to use nginx <link id="pool">pools</link>. A pool is created and tied to an object - <link id="http_conf">configuration</link>, <link id="cycle">cycle</link>, <link id="connection">connection</link>, or <link id="http_request">HTTP request</link>. When the object is destroyed, the associated pool is destroyed too. So when working with an object, it is possible to allocate the amount needed from the corresponding pool and don't care about freeing memory even in case of errors. </para> </section> <section name="Threads" id="threads_pitfalls"> <para> It is recommended to avoid using threads in nginx because it will definitely break things: most nginx functions are not thread-safe. It is expected that a thread will be executing only system calls and thread-safe library functions. If you need to run some code that is not related to client request processing, the proper way is to schedule a timer in the <literal>init_process</literal> module handler and perform required actions in timer handler. Internally nginx makes use of <link id="threads">threads</link> to boost IO-related operations, but this is a special case with a lot of limitations. </para> </section> <section name="Blocking Libraries" id="libraries"> <para> A common mistake is to use libraries that are blocking internally. Most libraries out there are synchronous and blocking by nature. In other words, they perform one operation at a time and waste time waiting for response from other peer. As a result, when a request is processed with such library, whole nginx worker is blocked, thus destroying performance. Use only libraries that provide asynchronous interface and don't block whole process. </para> </section> <section name="HTTP Requests to External Services" id="http_requests_to_ext"> <para> Often modules need to perform an HTTP call to some external service. A common mistake is to use some external library, such as libcurl, to perform the HTTP request. It is absolutely unnecessary to bring a huge amount of external (probably <link id="using_libraries">blocking</link>!) code for the task which can be accomplished by nginx itself. </para> <para> There are two basic usage scenarios when an external request is needed: <list type="bullet"> <listitem> in the context of processing a client request (for example, in content handler) </listitem> <listitem> in the context of a worker process (for example, timer handler) </listitem> </list> </para> <para> In the first case, the best is to use <link id="http_subrequests">subrequests API</link>. Instead of directly accessing external service, you declare a location in nginx configuration and direct your subrequest to this location. This location is not limited to <link doc="../http/ngx_http_proxy_module.xml" id="proxy_pass">proxying</link> requests, but may contain other nginx directives. An example of such approach is the <link doc="../http/ngx_http_auth_request_module.xml" id="auth_request"/> directive implemented in <link url="http://hg.nginx.org/nginx/file/tip/src/http/modules/ngx_http_auth_request_module.c">ngx_http_auth_request module</link>. </para> <para> For the second case, it is possible to use basic HTTP client functionality available in nginx. For example, <link url="http://hg.nginx.org/nginx/file/tip/src/event/ngx_event_openssl_stapling.c">OCSP module</link> implements simple HTTP client. </para> </section> </section> </article>