[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnunet-scheme] 06/07: doc: Split relatively large chapters into separat
|
From: |
gnunet |
|
Subject: |
[gnunet-scheme] 06/07: doc: Split relatively large chapters into separate files. |
|
Date: |
Sun, 06 Feb 2022 10:53:29 +0100 |
This is an automated email from the git hooks/post-receive script.
maxime-devos pushed a commit to branch master
in repository gnunet-scheme.
commit 973d5cc3398ff7581173c8c784cc281d7a838c15
Author: Maxime Devos <maximedevos@telenet.be>
AuthorDate: Sat Feb 5 20:24:59 2022 +0000
doc: Split relatively large chapters into separate files.
This seems a bit tidier to me.
* doc/scheme-gnunet.tm: Extract some chapters into ...
* doc/configuration.tm: ... a separate file.
* doc/contributing.tm: Likewise.
* doc/distributed-hash-table.tm: Likewise.
* doc/network-size-estimation.tm: Likewise.
* doc/network-structures.tm: Likewise.
* doc/service-communication.tm: Likewise.
* Makefile.am (tm_fragments): New variable.
(doc/sheme-gnunet.df,doc/scheme-gnunet.html): Add dependency
information.
(dist_doc_DATA): Install and distribute new files.
---
.reuse/dep5 | 2 +-
Makefile.am | 11 +-
doc/configuration.tm | 185 +++++++
doc/contributing.tm | 150 ++++++
doc/distributed-hash-table.tm | 270 ++++++++++
doc/network-size-estimation.tm | 81 +++
doc/network-structures.tm | 210 ++++++++
doc/scheme-gnunet.tm | 1138 +---------------------------------------
doc/service-communication.tm | 325 ++++++++++++
9 files changed, 1237 insertions(+), 1135 deletions(-)
diff --git a/.reuse/dep5 b/.reuse/dep5
index dc0df22..34abeaa 100644
--- a/.reuse/dep5
+++ b/.reuse/dep5
@@ -7,7 +7,7 @@ Files: doc/fdl.tm
Copyright: 2000 2001 2002 2007 2008 Free Software Foundation, Inc.
<https://fsf.org>
License: LicenseRef-Verbatim
-Files: doc/scheme-gnunet.tm
+Files: doc/configuration.tm doc/contributing.tm doc/distributed-hash-table.tm
doc/network-size-estimation.tm doc/network-structures.tm
doc/service-communication.tm doc/scheme-gnunet.tm
Copyright: 2021 2022 GNUnet e.V.
License: GFDL-1.3-no-invariants-or-later
diff --git a/Makefile.am b/Makefile.am
index 6144608..e7ac0f3 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -219,12 +219,17 @@ TEXMACS_HTML_SETTINGS = \
(set-boolean-preference "texmacs->html:mathjax" \#f)\
(set-boolean-preference "texmacs->html:mathml" \#t)'
-doc/scheme-gnunet.pdf: doc/scheme-gnunet.tm
+tm_fragments = \
+ doc/configuration.tm doc/contributing.tm doc/distributed-hash-table.tm \
+ doc/network-size-estimation.tm doc/network-structures.tm \
+ doc/service-communication.tm
+
+doc/scheme-gnunet.pdf: doc/scheme-gnunet.tm $(tm_fragments)
$(TEXMACS_CONVERT) -c "$<" "$@" -q
-doc/scheme-gnunet.html: doc/scheme-gnunet.tm
+doc/scheme-gnunet.html: doc/scheme-gnunet.tm $(tm_fragments)
$(TEXMACS_CONVERT) -x $(TEXMACS_HTML_SETTINGS) -c "$<" "$@" -q
-dist_doc_DATA = doc/scheme-gnunet.tm doc/fdl.tm
+dist_doc_DATA = doc/scheme-gnunet.tm $(tm_fragments) doc/fdl.tm
nobase_dist_doc_DATA = examples/web.scm
nodist_html_DATA = doc/scheme-gnunet.html
nodist_pdf_DATA = doc/scheme-gnunet.pdf
diff --git a/doc/configuration.tm b/doc/configuration.tm
new file mode 100644
index 0000000..3ac8261
--- /dev/null
+++ b/doc/configuration.tm
@@ -0,0 +1,185 @@
+<TeXmacs|2.1>
+
+<project|scheme-gnunet.tm>
+
+<style|tmmanual>
+
+<\body>
+ There are a number of modules for accessing GNUnet
+ configurations<index|configuration>.<space|1em>Firstly, there is <scm|(gnu
+ gnunet config db)><index|(gnu gnunet config db)>, which is the module
+ library code would typically use.<space|1em>For testing, one can create an
+ empty configuration with the procedure
<scm|hash-\<gtr\>configuration><index|hash-\<gtr\>configuration>
+ from that module and <scm|make-hashtable> from <scm|(rnrs hashtables)>,
+ using <scm|hash-key><index|hash-key> as hash function and
+ <scm|key=?><index|key=?> as comparison function:
+
+ <\scm-code>
+ (import (gnu gnunet config db)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet config value-parser)
+
+ \ \ \ \ \ \ \ \ (rnrs hashtables))
+
+ (define config (hash-\<gtr\>configuration (make-hashtable hash-key
+ key=?))
+ </scm-code>
+
+ The resulting configuration <scm|config> is initially empty, so set some
+ <with|font-shape|italic|keys> in the <with|font-shape|italic|section>
+ <verbatim|nse>, to configure the network-size estimation service:
+
+ <\scm-code>
+ (set-value! identity config "nse" "UNIXPATH"
+ "/tmp/nse.sock")<index|set-value!>
+
+ (set-value! number-\<gtr\>string config "cadet" "MAX_ROUTES" 5000)
+
+ ;; TODO: IP address, time durations, booleans, ...
+ </scm-code>
+
+ Now read these values back:
+
+ <\scm-code>
+ (read-value value-\<gtr\>file-name config "nse"
+ "UNIXPATH")<index|read-value>
+
+ ;; -\<gtr\> /tmp/nse.sock
+
+ (read-value value-\<gtr\>natural config "cadet" "MAX_ROUTES")
+
+ ;; -\<gtr\> 5000
+ </scm-code>
+
+ What if the configuration doesn't have a value for the specified section
+ and key?<space|1em>Then an
<scm|&undefined-key-error><index|&undefined-key-error>
+ results:
+
+ <\scm-code>
+ (read-value value-\<gtr\>natural config "kated" "MAX_ROUTES")
+
+ ;; -\<gtr\>
+
+ ;; ice-9/boot-9.scm:1685:16: In procedure raise-exception:
+
+ ;; ERROR:
+
+ ;; \ \ 1. &undefined-key-error:
+
+ ;; \ \ \ \ \ \ section: "kated"
+
+ ;; \ \ \ \ \ \ key: "MAX_ROUTES"
+ </scm-code>
+
+ <section|Locating configuration files>
+
+ There are two \U possibly non-existent \U configuration files: the
+ <with|font-shape|italic|user> configuration<subindex|configuration|user>
+ and the <with|font-shape|italic|system>
+ configuration<subindex|configuration|system>.<space|1em>The
+ <with|font-shape|italic|system> configuration typically contains the paths
+ for services like NSE, CORE, <text-dots><space|1em>A user may choose not to
+ use the services by the system and instead use their own.<space|1em>To do
+ so, the user needs to override the paths in the
+ <with|font-shape|italic|user> configuration.<space|1em><todo|defaults?> The
+ module <scm|(gnu gnunet config fs)><index|(gnu gnunet config fs)> is
+ responsible for determining the location of the configuration files to load
+ and actually load configuration files.<space|1em>For determining the
+ location of the configuration files, the procedures
+ <scm|locate-user-configuration> and <scm|locate-system-configuration> can
+ be used.
+
+ <\warning>
+ The C implementation's mechanism for user-system separation seems to work
+ differently.
+ </warning>
+
+ <\explain>
+ <scm|(locate-user-configuration
<scm|#:getenv=getenv>)><index|locate-user-configuration>
+ <|explain>
+ This procedure determines the location of the user configuration file, as
+ a string, or <scm|#false> if it could not be determined.<space|1em>If the
+ location of the user configuration file is known, but the file does not
+ exist, it is returned anyway, as a string.
+
+ If the environment variable <shell|XDG_CONFIG_HOME> is set, the location
+ of the file <verbatim|gnunet.conf> in the directory
+ <shell|$XDG_CONFIG_HOME> is returned.<space|1em>If the environment
+ variable is not set, the location of the file at
+ <verbatim|.config/gnunet.conf><index|~/.config/gnunet.conf> in the home
+ directory specified by the environment variable <verbatim|HOME> is
+ returned, if that environment variable exists.<space|1em>If both are
+ unset, <scm|#false> is returned.
+
+ The values of environment variables is determined with the procedure
+ <scm|getenv>.
+ </explain>
+
+ <\explain>
+ <scm|(locate-system-configuration)><index|locate-system-configuration>
+ <|explain>
+ This procedure determines the location of the system configuration file,
+ as a string.
+
+ Currently, this is always
<verbatim|/etc/gnunet.conf><index|/etc/gnunet.conf>.
+ </explain>
+
+ <section|Loading configuration files>
+
+ Once the location of the configuration file is known, the file can be
+ opened with the Scheme procedure <scm|open-input-file>, which returns an
+ input port.<space|1em>Then the procedure <scm|load-configuration/port!> can
+ be used to determine all section-key-values triples in the configuration.
+
+ <\explain>
+ <scm|(load-configuration/port! <var|set-value!>
+ <var|port>)><index|load-configuration/port!>
+ <|explain>
+ Load the configuration from the input port <var|port>.
+
+ For each variable, call <var|set-value!> with the section name, variable
+ name and a vector of the form <scm|#(line line-number value)>, where
+ <var|value> a list of expansible objects.
+
+ <todo|document expansible objects><todo|error reporting>
+ </explain>
+
+ A variable assignment<index|variable assignment> <verbatim|[section]
+ key=value${var}> can refer to variables defined in the <verbatim|PATHS>
+ section and variables from the environment.<space|1em>The previously
+ described procedure <scm|load-configuration/port!> will <em|not> expand
+ such assignements. \ To expand variable assignments, use the procedure
+ <scm|make-expanded-configuration> instead.
+
+ <\explain>
+ <scm|(make-expanded-configuration <var|load!>
+ #:getenv=<var|getenv>)><index|make-expanded-configuration>
+ <|explain>
+ Make a configuration object.<space|1em>To populate the configuration, all
+ the procedure <var|load!> with a <scm|set-value!> procedure as expected
+ by <scm|load-configuration/port!>.<space|1em>The values from
+ <scm|set-value!> are added to the confoiguration and every variable is
+ expanded.
+ </explain>
+
+ To automatically load the defaults, the system configuration and the user
+ configuration, use the thunk <scm|load-configuration>:
+
+ <\explain>
+ <scm|(load-configuration #:getenv=<var|getenv>
+ #:files=<text-dots>)><index|load-configuration>
+ </explain|Load the defaults, the system configuration and the user
+ configuration and return the resulting configuration object.<space|1em>The
+ list of files to load can be overriden by setting the undocumented
+ <var|files> keyword argument.>
+
+ Applications (whether graphical or textual) are recommended to use
+ <scm|load-configuration> by default, as it largely just works.
+</body>
+
+<\initial>
+ <\collection>
+ <associate|page-medium|paper>
+ <associate|save-aux|false>
+ </collection>
+</initial>
\ No newline at end of file
diff --git a/doc/contributing.tm b/doc/contributing.tm
new file mode 100644
index 0000000..724530f
--- /dev/null
+++ b/doc/contributing.tm
@@ -0,0 +1,150 @@
+<TeXmacs|2.1>
+
+<project|scheme-gnunet.tm>
+
+<style|generic>
+
+<\body>
+ <section|Building from source>
+
+ The latest \<#2018\>official\<#2019\> development version of scheme-GNUnet
+ can be found at
<hlink|https://git.gnunet.org/gnunet-scheme.git/|https://git.gnunet.org/gnunet-scheme.git/>.
+ It can be downloaded with <hlink|git|https://en.wikipedia.org/wiki/Git>.
+ The following software needs to be installed first:
+
+ <\itemize>
+ <item>The Autotools
(<hlink|autoconf|https://www.gnu.org/software/autoconf/>
+ and <hlink|automake|https://www.gnu.org/software/automake/>)
+
+ <item><hlink|GNU Guile|https://www.gnu.org/software/guile/> (at least
+ version 3), with the patch from
<slink|https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49623>
+ applied
+
+ <item><hlink|Purely Functional Data Structures in Scheme
+ (pfds)|https://github.com/ijp/pfds/>
+
+ <item><hlink|(Guile) Fibers|https://github.com/wingo/fibers/>, with the
+ <slink|https://github.com/wingo/fibers/pull/50> patch
+
+
<item><hlink|Guile-QuickCheck|https://ngyro.com/software/guile-quickcheck.html>
+
+ <item><hlink|Guile-Gcrypt|https://notabug.org/cwebber/guile-gcrypt>
+ </itemize>
+
+ Users of <hlink|GNU Guix|https://guix.gnu.org><index|Guix> can run
+ <shell|guix shell -D gnunet-scheme> in the checkout to create an
+ environment where these dependencies are all
+ present.<space|1em>Scheme-GNUnet uses the standard GNU build system, so to
+ build Scheme-Gnunet, you only need to run
+
+ <\shell-code>
+ autoreconf -vif
+
+ ./configure
+
+ make
+
+ make check
+ </shell-code>
+
+ After building, the documentation is available at
+ <verbatim|doc/scheme-gnunet.pdf> and <verbatim|doc/scheme-gnunet.html> in
+ PDF and HTML formats.<space|1em>To get started, you can run the example
+ mini-application at <verbatim|examples/nse-web.scm> and point your browser
+ at <slink|http://localhost:8089>:
+
+ <\shell-code>
+ $ guile -L . -C . -l examples/nse-web.scm
+ </shell-code>
+
+ <subsection|Authenticating new source code><index|authentication>
+
+ When GNU Guix is present, after pulling the latest Scheme-GNUnet commit,
+ the following command can be run to verify it is authentic:
+
+ <\shell-code>
+ guix git authenticate 088ea8b27b95143584cbc36202c5a02dfe95796c "C1F3 3EE2
+ 0C52 8FDB 7DD7 \ 011F 49E3 EE22 1917 25EE"
+ </shell-code>
+
+ If it isn't authentic, an error message such as the following will be
+ written:
+
+ <\shell-code>
+ Authenticating commits 54a74dc to 431f336 (1 new commits)...
+
+ [#########################################################]
+
+ guix git: error: commit 431f336edd51e1f0fe059a6f6f2d4c3e9267b7bc not
+ signed by an authorized key: C1F3 3EE2 0C52 8FDB 7DD7 \ 011F 49E3 EE22
+ 1917 25EE
+ </shell-code>
+
+ <section|Writing tests>
+
+ <index|tests>\<#2018\><hlink|How SQLite Is
+ Tested|https://sqlite.org/testing.html>\<#2019\> is a recommended
+ read.<space|1em>Scheme-GNUnet isn't that well-tested but still aims for
+ being free of bugs and having many tests to prevents bugs from being
+ introduced.<space|1em>When adding new code, consider writing test
+ cases.<space|1em>Some things that can be tested and few methods for testing
+ things:
+
+ <\itemize>
+ <item>Run mutation tests.<space|1em>That is, replace in the source code
+ <scm|\<less\>> with <scm|\<less\>=>, <scm|0> with <scm|1>, a variable
+ reference <scm|i> with a variable reference <scm|j>, swap destination and
+ source arguments <text-dots> and verify whether the tests detect these
+ little mutations.
+
+ <item>Be exhaustive.<space|1em>If a procedure handles both foos and bars,
+ write test cases that pass the procedure a foo and test cases that pass
+ the procedure a bar.<space|1em>Sometimes Guile-QuickCheck can help with
+ generating many test cases if the input has a regular structure yet many
+ edge cases, see e.g. <verbatim|tests/cmsg.scm>.
+
+ <item>Verify exception mechanisms!<space|1em>If a procedure is expected
+ to handle I/O errors, simulate I/O errors and end-of-files in all the
+ wrong places.<space|1em>If the procedure can raise exceptions, make sure
+ these exceptions are raised when necessary.
+ </itemize>
+
+ Tests are added in the directory <scm|tests> and to the variable
+ <verbatim|SCM_TESTS> in <verbatim|Makefile.am> and use <scm|srfi
+ :64>.<space|1em>To run the test suite, run <verbatim|make check>.
+
+ <section|Contact>
+
+ Scheme-GNUnet is currently maintained on NotABug:
+ <slink|https://notabug.org/maximed/scheme-gnunet/>.<space|1em>Issues and
+ pull requests can be reported and submitted here.<space|1em>Alternatively,
+ for discussion about developing Scheme-GNUnet, you can send mails to
+ <hlink|gnunet-devel@gnu.org|mailto:gnunet-devel@gnu.org> and for help about
+ how to use Scheme-GNUnet, you can contact
+ <hlink|help-gnunet@gnu.org|mailto:help-gnunet@gnu.org>.<space|1em>These are
+ public mailing lists, so don't send anything there you wouldn't mind the
+ whole world to know.
+
+ For security-sensitive issues, you can send a mail directly to the
+ maintainer, <hlink|Maxime Devos
\<less\>maximedevos@telenet.be\<gtr\>|mailto:maximedevos@telenet.be>,
+ optionally encrypted and signed with a GnuPG-compatible
+ system.<space|1em>The maintainer's key fingerprint is C1F3 3EE2 0C52 8FDB
+ 7DD7 011F 49E3 EE22 1917 25EE and a copy of the key can be downloaded from
+ <slink|https://notabug.org/maximed/things/raw/master/Maxime_Devos.pub>.
+
+ <section|License>
+
+ The code of Scheme-GNUnet is available under the Affero General Public
+ License (AGPL), version 3 or later; see individual source files for
+ details.<space|1em>The documentaton is available under the GNU Free
+ Documentation License, see the start of this manual and the likewise-named
+ appendix for details.<space|1em>The AGPL has some unusual conditions w.r.t.
+ applications interacting with the network, please read it carefully.
+</body>
+
+<\initial>
+ <\collection>
+ <associate|page-medium|paper>
+ <associate|save-aux|false>
+ </collection>
+</initial>
\ No newline at end of file
diff --git a/doc/distributed-hash-table.tm b/doc/distributed-hash-table.tm
new file mode 100644
index 0000000..b9f25fe
--- /dev/null
+++ b/doc/distributed-hash-table.tm
@@ -0,0 +1,270 @@
+<TeXmacs|2.1>
+
+<project|scheme-gnunet.tm>
+
+<style|tmmanual>
+
+<\body>
+ GNUnet has a service that maintains a <em|distributed hash
+ table><index|distributed hash table><index|DHT><index|R5N>, mapping keys to
+ values. The module <scm|(gnu gnunet dht client)><index|(gnu gnunet dht
+ client)> can be used to interact with the service. The connection can be
+ made with the procedure <scm|connect>. It returns a <em|DHT server
+ object><index|DHT server object><subindex|server object|DHT>.
+
+ <\explain>
+ <scm|(connect <var|config> <var|#:connected> <var|#:disconnected>
+ <var|#:spawn>)><subindex|connect|DHT>
+ <|explain>
+ Connect to the DHT service, using the configuration <var|config>. The
+ connection is made asynchronuously; the optional thunk <var|connected> is
+ called when the connection has been made. The connection can break; the
+ optional thunk <var|disconnected> is called when it does. If the
+ connection breaks, the client code automatically tries to reconnect, so
+ <var|connected> can be called after <var|disconnected>.
+ </explain>
+
+ <\explain>
+ <scm|(disconnect! <var|server>)><subindex|disconnect!|DHT>
+ <|explain>
+ Asynchronuously disconnect from the DHT service and stop reconnecting,
+ even if not connected. This is an idempotent operation.
+ </explain>
+
+ <section|Data in the DHT>
+
+ To insert data into the DHT, the DHT service needs various information \U
+ the key and the value, but also some other information. Likewise, when the
+ DHT service has found the datum, this information is available as well. A
+ datum in the DHT is represented as a <em|datum object><index|datum object>.
+ The object holding the information required for inserting something in the
+ DHT is called an <em|insertion object><index|insertion object>. Likewise,
+ the information of searching for something in the DHT is in a <em|query
+ object><index|query object>. The result of a query is a <em|search result
+ object><index|search result object>.
+
+ These objects only hold information; creating them does not have any side
+ effects.
+
+ <\explain>
+ <scm|(make-datum <var|type> <var|key> <var|value>
+ <var|#:expiration>)><index|make-datum>
+ <|explain>
+ Make a datum object of block type <var|type> (or its corresponding
+ numeric value), with key <var|key> (a readable <scm|/hashcode:512>
+ bytevector slice), value <var|value> (a readable bytevector slice) and
+ expiring at <var|expiration> (<todo|type, epoch>). The keyword argument
+ <var|expiration> is optional, see <reference|???>.
+
+ The numeric value of the block type can be retrieved with the accessor
+ <scm|datum-type>. The accessors <scm|datum-key><index|datum-key>,
+ <scm|datum-value><index|datum-value> and
+ <scm|datum-expiration><index|datum-expiration> return the key, value and
+ expiration time respectively. It can be tested if an object is a datum
+ object with the predicate <scm|datum?><index|datum?>.
+
+ The length of <var|value> may be at most
+ <scm|%max-datum-value-length><index|%max-datum-value-length>. If this
+ bound is exceeded, an appropriate
<scm|&overly-large-datum><index|&overly-large-datum>
+ and <scm|&who> condition is raised.
+ </explain>
+
+ <\explain>
+ <scm|(datum-\<gtr\>insertion <var|datum>
+ #:desired-replication-level)><index|datum-\<gtr\>insertion>
+ <|explain>
+ Make an insertion object for inserting the datum <var|datum>, desiring a
+ replication level <var|desired-replication-level> (see
+ <reference|replication levels???>)<todo|various options>.
+
+ The datum and desired replication level can be recovered with the
+ accessors <scm|insertion-\<gtr\>datum><index|insertion-\<gtr\>datum> and
+
<var|insertion-desired-replication-level><index|insertion-desired-replication-level>.
+ It can be tested if an object is an insertion object with the predicate
+ <scm|insertion?><index|insertion?>.
+ </explain>
+
+ <\explain>
+ <scm|(make-query <var|type> <var|key>
+ #:desired-replication-level)><index|make-query>
+ <|explain>
+ Make a query object for searching for a value of block type <var|type>
+ (or its corresponding numeric value), with key <var|key> (a readable
+ <scm|/hashcode:512> bytevector slice), at desired replication level
+ <scm|desired-replication-level> (see <reference|replication levels???>).
+ <todo|various options, xquery>
+
+ The numeric value of the block type, the key and the desired replication
+ level can be recovered with the accessors
+ <scm|query-type><index|query-type>, <scm|query-key><index|query-key> and
+
<scm|query-desired-replication-level><index|query-desired-replication-level>.
+ It can be tested if an object is a query object with the predicate
+ <scm|query?><index|query?>.
+ </explain>
+
+ <\explain>
+ <scm|(datum-\<gtr\>search-result <var|datum> #:get-path
+ #:put-path)><index|datum-\<gtr\>search-result>
+ <|explain>
+ Make a search result object for the datum <var|datum>. The datum can be
+ recovered with the accessor
<scm|search-result-\<gtr\>datum><index|search-result-\<gtr\>datum>.
+ It can be tested if an object is a search result with the predicate
+ <scm|search-result?><index|search-result?>. The optional arguments
+ <var|get-path> and <var|put-path>, when not false, are bytevector slices
+ consisting of a list of
<scm|/dht:path-element><index|/dht:path-element><index|path
+ element>.
+
+ The <var|get-path><index|get path> , if any, is the path from the storage
+ location to the current peer. Conversely, the <var|put-path><index|put
+ path>, if any, is a path from the peer that inserted the datum into the
+ DHT to the storage location. The <var|get-path> and <var|put-path> can be
+ accessed with <scm|search-result-get-path><index|search-result-get-path>
+ and <scm|search-result-put-path><index|search-result-put-path>
+ respectively.
+
+ When the datum, get path and put path together are too large, a
+ <scm|&overly-large-paths><index|&overly-large-paths> condition is raised.
+ When the bytevector slice length of <var|get-path> or <var|put-path> is
+ not a multiple of the size of a path element, then a
+ <scm|&malformed-path><index|&malformed-path> condition is raised.
+ </explain>
+
+ <section|Accessing data in the DHT>
+
+ To insert a datum into the DHT, the procedure <scm|put!> is used. To find
+ data matching a query, the procedure <scm|start-get!> is
+ used.<index|searching the DHT><index|inserting data into the DHT>
+
+ <\explain>
+ <scm|(start-get! <var|server> <var|query> <var|found>)><index|start-get!>
+ <|explain>
+ Search for data matching <var|query> in the DHT. When a datum is found,
+ call the unary procedure <var|found> on the search result. It is possible
+ to find multiple data matching a query. In that case, <var|found> is
+ called multiple times. Searching happens asynchronuously; to stop the
+ search, a fresh <em|search object><index|search object> for controlling
+ the search is returned.
+
+ The procedure <var|found> is run from the context of <var|server>. As
+ such, if <var|found> blocks, then all operations on <var|server> might
+ block. As such, it is recommended for <var|found> to do as little as
+ possible by itself and instead delegate any work to a separate fiber.
+
+ To avoid expensive copies, the implementation can choose to reuse
+ internal buffers for the slices passed to <var|found>, which could be
+ overwritten after the call to <var|found>. As such, it might be necessary
+ to make a copy of the search result, using <scm|copy-search-result>.
+ </explain>
+
+ <\explain>
+ <scm|(put! <var|server> <var|insertion> <var|#:confirmed>)><index|put!>
+ <|explain>
+ Perform the insertion <var|insertion>. When the datum has been inserted,
+ the optional thunk <var|confirmed> is called. A <em|put object> is
+ returned which can be used to cancel the insertion.
+
+ <todo|TODO: actually call <var|confirmed>>
+ </explain>
+
+ <\explain>
+ <scm|(copy-query <var|old>)><index|copy-query>
+
+ <scm|(copy-datum <var|old>)><index|copy-datum>
+
+ <scm|(copy-insertion <var|old>)><index|copy-insertion>
+
+ <scm|(copy-search-result <var|old>)><index|copy-search-result>
+ <|explain>
+ Make a copy of the object <var|old> (a query, datum, insertion or search
+ result object, depending on the procedure), such that modifications to
+ the slices in <var|old> do not impact the new object.
+ </explain>
+
+ <todo|cancellation>
+
+ <section|Constructing and analysing network messages>
+
+ The DHT client and service communicate by sending <em|messages>. Usually,
+ only the implementation of the client and service need to construct and
+ analyse these messages, but nothing prevents other uses of the procedures
+ in <scm|(gnu gnunet dht network)><index|(gnu gnunet dht network)>, e.g. for
+ learning, in a tool like Wireshark or for tests.
+
+ The <em|analysis> procedures<index|analysis procedures> assume that the
+ message is well-formed and avoid constructing new bytevector slices by
+ taking subslices. The <em|construction> procedures<index|construction
+ procedures> create fresh well-formed read-write bytevector slices.
+
+ <\warning>
+ Possibly the type of <var|options> will change and possibly the options
+ will be moved into the query object and insertion object.
+ </warning>
+
+ <\explain>
+ <scm|(construct-client-get <var|query> <var|unique-id> #:optional
+ (<var|options> 0))><index|construct-client-get>
+ <|explain>
+ Create a new <scm|/:msg:dht:client:get><index|/:msg:dht:client:get>
+ message for the query object <var|query>, with <var|unique-id> as
+ \<#2018\>unique id\<#2019\> and <var|options> as options.
+ </explain>
+
+ <\explain>
+ <scm|(construct-client-put <var|insertion> #:optional (options
+ 0))><index|construct-client-put>
+ <|explain>
+ Create a new <scm|/:msg:dht:client:put><index|/:msg:dht:client:put>
+ message for the insertion object <var|insertion> with <var|options> as
+ options.
+ </explain>
+
+ <\explain>
+ <scm|(construct-client-result <var|search-result>
+ <var|unique-id>)><index|construct-client-result>
+ <|explain>
+ Create a new <scm|/:msg:dht:client:result><index|/:msg:dht:client:result>
+ message for the search result object <var|search-result>, with
+ <var|unique-id> as \<#2018\>unique id\<#2019\> .
+ </explain>
+
+ <\explain>
+ <scm|(analyse-client-get <var|message>)><index|analyse-client-get>
+ <|explain>
+ Return the query object, the unique id and the options corresponding to
+ the <scm|/:msg:dht:client:result><index|/:msg:dht:client:result> message
+ <var|message>. Xqueries are currently unsupported.
+ </explain>
+
+ <\explain>
+ <scm|(analyse-client-put <var|message>)><index|analyse-client-put>
+ <|explain>
+ Return the insertion object and options corresponding to the
+ <scm|/:msg:dht:client:put><index|/:msg:dht:client:put> message
+ <var|message>.
+ </explain>
+
+ <\explain>
+ <scm|(analyse-client-result <var|message>)><index|analyse-client-result>
+ <|explain>
+ Return search result object and unique id for the
+ <scm|/:msg:dht:client:result><index|/:msg:dht:client:result> message
+ <var|message>.
+ </explain>
+
+ <todo|monitoring messages>
+
+ <section|How to handle invalid data>
+
+ <todo|todo!>
+
+ <section|Monitoring: spying on what other applications and peers are doing>
+
+ <todo|todo!>
+</body>
+
+<\initial>
+ <\collection>
+ <associate|page-medium|paper>
+ <associate|save-aux|false>
+ </collection>
+</initial>
\ No newline at end of file
diff --git a/doc/network-size-estimation.tm b/doc/network-size-estimation.tm
new file mode 100644
index 0000000..2be4361
--- /dev/null
+++ b/doc/network-size-estimation.tm
@@ -0,0 +1,81 @@
+<TeXmacs|2.1>
+
+<project|scheme-gnunet.tm>
+
+<style|tmmanual>
+
+<\body>
+ <index|network size estimation><index|NSE>GNUnet has a service that roughly
+ estimates the size of the network \U i.e., the number of
+ peers.<space|1em>The module <scm|(gnu gnunet nse client)><index|(gnu gnunet
+ nse client)> can be used to interact with this service.<space|1em>The
+ connection is made with the procedure <scm|connect><subindex|connect|NSE>,
+ which is accepts a <with|font-shape|italic|configuration> (see
+ <todo|reference>) and some optional keyword arguments.<space|1em>This
+ procedure can be called as <scm|(connect config #:updated updated
+ #:connected connected #:disconnected disconnected)>.<space|1em>It returns a
+ <with|font-shape|italic|NSE server object><index|NSE server
+ object><subindex|server object|NSE>.
+
+ The connection is made asynchronuously; the thunk <var|connected> will be
+ called when the connection has actually been made.<space|1em>Whenever a new
+ estimate becomes available, the (optional) procedure
+ <var|updated><index|update procedure> is called with the new
+ <with|font-shape|italic|estimate><index|estimate
+ object>.<space|1em>Alternatively, the procedure
+ <scm|estimate><index|estimate> can be called on the server object to return
+ the latest available estimate.<space|1em>If the
+ <with|font-shape|italic|server object> doesn't have an estimate yet, that
+ procedure will return <scm|#false> instead of an estimate.
+
+ When the connection is lost, the (optional) thunk <var|disconnected> is
+ called and <scm|(gnu gnunet nse client)> will retry
+ connecting.<space|1em>To close the current connection, if any, and stop
+ reconnecting, the idempotent procedure
<scm|disconnect!><subindex|disconnect!|NSE>
+ can be called on the server object.
+
+ <todo|input, validation, I/O errors?>
+
+ The estimate object has a number of accessors:
+
+ <\explain>
+ <scm|(estimate:logarithmic-number-peers
+ <var|estimate>)><index|estimate:logarithmic-number-peers>
+ </explain|The base-2 logarithm of the number of peers (estimated), as a
+ positive flonum, possibly zero or infinite>
+
+ <\explain>
+ <scm|(estimate:number-peers <var|estimate>)><index|estimate:number-peeers>
+ </explain|The number of peers (estimated), as a flonum, at least <scm|1.0>
+ and possibly infinite.<space|1em>This is not necessarily an (inexact)
+ <scm|integer?>, as it is only an estimate.>
+
+ <\explain>
+ <scm|(estimate:timestamp estimate)><index|estimate:timestamp>
+ </explain|A timestamp when the estimate was made <todo|something about
+ epoch?>>
+
+ <\explain>
+ <scm|(estimate:standard-deviation
<var|estimate>)><index|estimate:standard-deviation>
+ <|explain>
+ The estimated standard deviation on the base-2 logarithm of peers,
+ calculated over the last 64 rounds, with the <math|<frac|N|N-1>>
+ correction.<space|1em>This is a positive flonum, possibly zero or
+ infinite, or not-a-number (indicating division of zero by zero).
+
+ If the peer has been connected to the network for a while, the number is
+ expected to be finite and strictly positive.
+ </explain>
+
+ Assuming the network size is stable and the errors on the logarithmic
+ estimate are normally distributed, the procedure
+ <scm|estimate:standard-deviation> can be used to put probablistic error
+ bounds on the number of peers on the network. <todo|example>
+</body>
+
+<\initial>
+ <\collection>
+ <associate|page-medium|paper>
+ <associate|save-aux|false>
+ </collection>
+</initial>
\ No newline at end of file
diff --git a/doc/network-structures.tm b/doc/network-structures.tm
new file mode 100644
index 0000000..9a9941c
--- /dev/null
+++ b/doc/network-structures.tm
@@ -0,0 +1,210 @@
+<TeXmacs|2.1>
+
+<project|scheme-gnunet.tm>
+
+<style|tmmanual>
+
+<\body>
+ <index|network structure><index|netstruct>The modules <scm|(gnu gnunet
+ netstruct procedural)> and <scm|(gnu gnunet netstruct syntactic)> can be
+ used for formatting messages to be sent over the network or to a
+ service.<space|1em>The macros <scm|define-type><index|define-type> and
+ <scm|structure/packed><index|structure/packed> can be used to define new
+ structures, like this:
+
+ <\scm-code>
+ (define-type /:msg:nse:estimate/example
+
+ \ \ (structure/packed
+
+ \ \ \ (properties '((message-symbol msg:nse:estimate)))
+
+ \ \ \ (synopsis "Network size estimate")
+
+ \ \ \ (documentation "Some explanation")
+
+ \ \ \ (field (header /:message-header))
+
+ \ \ \ (field (size-estimate ieee-double/big)
+
+ \ \ \ \ \ \ \ \ \ \ (synopsis "Timestamp for when the estimate was
+ made"))))
+ </scm-code>
+
+ This example is taken from the <scm|(gnu gnunet nse struct)> module and
+ oversimplified.<space|1em>All its components will gradually
+ explained.<space|1em>First, what actually is a network
+ structure?<space|1em>This question is ambigious, because \<#2018\>network
+ structure\<#2019\> can refer to either the <with|font-shape|italic|value>
+ or the <with|font-shape|italic|type>.<space|1em>The
+ <with|font-shape|italic|value> is a sequence of octets, i.e., a sequence of
+ numbers in the closed range 0\U255.<space|1em>The
+ <with|font-shape|italic|type> describes how the
+ <with|font-shape|italic|value> is structured.
+
+ As an example, consider figure <reference|networkstructex>.<space|1em>There,
+ the value is <scm|0 12 1 65 64 51 238 123 71 27 58 149> and the type is
+ <scm|/:msg:nse:estimate/example>.
+
+ <big-figure|<tree|<scm|/:msg:nse:estimate/example>|<tree|<scm|header>
+ <scm|/:message-header>|<tree|<scm|size> <scm|u16/big>|0
+ 12>|<tree|<scm|type> <scm|u16/big>|1 65>>|<tree|<scm|size-estimate>
+ <scm|ieee-double/big>|64 51 238 123 71 27 58 149>>|A network structure,
+ both <with|font-shape|italic|value> and
+ <with|font-shape|italic|type>.<label|networkstructex>>
+
+ This value has a <with|font-shape|italic|header> with a
+ <with|font-shape|italic|size> \<#2018\>0 0 0 12\<#2019\> of type
+ <scm|u32/big>, so the <with|font-shape|italic|size> is 12.<space|1em>The
+ <with|font-shape|italic|header> also has a <with|font-shape|italic|type>
+ \<#2018\>0 0 1 65\<#2019\> of type <scm|u32/big>, so the type is
+ <math|256\<times\>1+65=321>.<space|1em>The value also has a
+ <with|font-shape|italic|size estimate> of type
+ <scm|ieee-double/big>.<space|1em>The octets <scm|64 51 238 123 71 27 58
+ 149>, \ interpreted as an IEEE double, form the number
+ <math|19.93\<ldots\>>.
+
+ <section|Documentation>
+
+ A network structure can optionally have embedded
+ documentation.<space|1em>More specifically, network structures can
+ optionally have the <with|font-shape|italic|synopsis><index|synopsis>,
+ <with|font-shape|italic|documentation><index|documentation> and
+ <with|font-shape|italic|properties><index|properties> set.<space|1em>The
+ <with|font-shape|italic|synopsis> is a short description of what the
+ network structure represents, typically a one-liner.<space|1em>The
+ <with|font-shape|italic|documentation> can be more detailed, explaining how
+ the structure works, can be used, is used and should be used.<space|1em>The
+ <with|font-shape|italic|properties> form a free-formed association list.
+
+ The synopsis, documentation and properties can be set on structured created
+ with <scm|structure/packed> and individual fields and can be accessed with
+ the procedures <scm|documentation>, <scm|synopsis> and <scm|properties>.
+
+ The following properties are defined in Scheme-GNUnet:
+
+ <\description>
+ <\item*>
+ <\code>
+ message-symbol
+ </code>
+ </item*>
+
+ <index|message-symbol>For <with|font-shape|italic|message structures>,
+ this is a list of the names of the <with|font-shape|italic|message types>
+ (see <reference|sec:message type> Message type database) this structure
+ can be used for \U most of the time, there's only a single such message
+ type, but sometimes a single structure can be used for multiple message
+ types.
+
+ <item*|c-type>
+
+ <label|c-type>The value is the name of the equivalent type in the C
+ implementation, if any, as a symbol. This can be useful if you know the
+ name of the C type but not the name of the Scheme type: in that case, you
+ can do <shell|git grep -F GNUNET_MessageHeader> in a git checkout of the
+ source code of Scheme-GNUnet to discover that Scheme-GNUnet's name for
+ <cpp|GNUNET_MessageHeader> is <scm|/:message-header>.
+ </description>
+
+ <todo|TODO: it would be nice to use the properties and documentation to
+ automatically create a form of documentation, with some cross-references to
+ the C code>
+
+ <section|Reading and writing>
+
+ The procedures <scm|read%><index|read%>, <scm|set%!><index|set%!>,
+ <scm|sizeof><index|sizeof> and <scm|select><index|select> from <scm|(gnu
+ gnunet netstruct procedural)> or the like-named macros from <scm|(gnu
+ gnunet netstruct syntactic)> can be used for reading and writing network
+ structures.<space|1em>The macros from <scm|syntactic> behave like the
+ procedures from <scm|procedural> with some optimisations at expansion
+ time.<space|1em>The procedures will be demonstrated with the
+ <scm|/:msg:nse:estimate/example> network structure defined previously.
+
+ First, create a memory slice with <scm|make-slice/read-write> from
+ <scm|(gnu gnunet utils bv-slice)>.<space|1em>The required size can be
+ determinded with <scm|(sizeof /:msg:nse:estimate/example
+ '())>.<space|1em>The role of <scm|'()> will be explained later.
+
+ <\scm-code>
+ (import (gnu gnunet netstruct procedural)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet util struct))
+
+ \;
+
+ (define-type /:msg:nse:estimate/example
+
+ \ \ (structure/packed
+
+ \ \ \ (field (header /:message-header))
+
+ \ \ \ (field (size-estimate ieee-double/big))))
+
+ \;
+
+ (define message
+
+ \ \ (make-slice/read-write (sizeof /:msg:nse:estimate/example '())))
+ </scm-code>
+
+ The fields of <scm|message> can be set with <scm|(set%! netstruct '(field
+ ...) slice value)>.<space|1em>The following code sets all the fields:
+
+ <\scm-code>
+ (set%! /:msg:nse:estimate/example '(header size) message
+
+ \ \ \ \ \ \ \ (sizeof /:msg:nse:estimate/example '()))
+
+ (set%! /:msg:nse:estimate/example '(header type) message 165)
+
+ (set%! /:msg:nse:estimate/example '(size-estimate) message 19.2)
+ </scm-code>
+
+ The size of an individual field can be determined with <scm|(sizeof
+ netstruct '(field ...))>.<space|1em>For example, the following code
+ determines the size of the \<#2018\>size\<#2019\> field in the header:
+
+ <\scm-code>
+ (sizeof /:msg:nse:estimate/example '(header size)) ; 12
+ </scm-code>
+
+ The fields can also be read:
+
+ <\scm-code>
+ (read% /:msg:nse:estimate/example '(header size) message) ; 12
+
+ (read% /:msg:nse:estimate/example '(header type) message) ; 165
+
+ (read% /:msg:nse:estimate/example '(size-estimate) message) ; 19.2
+ </scm-code>
+
+ <section|Primitive types>
+
+ There are a number of pre-defined types.<space|1em>First, there is
+ <scm|u8>, a single octet that is interpreted as an integer in the closed
+ range <math|<around*|[|0,255|]>>.<space|1em>There are also types
+ <scm|uN/endian> for <math|N\<in\><around*|{|16,32,64|}>> and
+ <math|endian\<in\><around*|{|little,big|}>>, which interprets <math|N/8>
+ octets as integers in the closed range
<math|<around*|[|0,2<rsup|N>-1|]>>.<space|1em>The
+ types <scm|ieee-double/big> and <scm|ieee-double/little> are 8 octets long
+ and represent floating-point numbers in IEEE 754 format
+ (\<#2018\>binary64\<#2019\>).
+
+ <section|Packing>
+
+ In contrast to C structures, Scheme-GNUnet network structures are always
+ packed \V there are no \<#2018\>gaps\<#2019\> between fields.
+</body>
+
+<\initial>
+ <\collection>
+ <associate|page-medium|paper>
+ <associate|save-aux|false>
+ </collection>
+</initial>
\ No newline at end of file
diff --git a/doc/scheme-gnunet.tm b/doc/scheme-gnunet.tm
index e7b9570..0f73fd1 100644
--- a/doc/scheme-gnunet.tm
+++ b/doc/scheme-gnunet.tm
@@ -202,141 +202,7 @@
<chapter|Installation and contributing guide>
- <section|Building from source>
-
- The latest \<#2018\>official\<#2019\> development version of scheme-GNUnet
- can be found at
<hlink|https://git.gnunet.org/gnunet-scheme.git/|https://git.gnunet.org/gnunet-scheme.git/>.
- It can be downloaded with <hlink|git|https://en.wikipedia.org/wiki/Git>.
- The following software needs to be installed first:
-
- <\itemize>
- <item>The Autotools
(<hlink|autoconf|https://www.gnu.org/software/autoconf/>
- and <hlink|automake|https://www.gnu.org/software/automake/>)
-
- <item><hlink|GNU Guile|https://www.gnu.org/software/guile/> (at least
- version 3), with the patch from
<slink|https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49623>
- applied
-
- <item><hlink|Purely Functional Data Structures in Scheme
- (pfds)|https://github.com/ijp/pfds/>
-
- <item><hlink|(Guile) Fibers|https://github.com/wingo/fibers/>, with the
- <slink|https://github.com/wingo/fibers/pull/50> patch
-
-
<item><hlink|Guile-QuickCheck|https://ngyro.com/software/guile-quickcheck.html>
-
- <item><hlink|Guile-Gcrypt|https://notabug.org/cwebber/guile-gcrypt>
- </itemize>
-
- Users of <hlink|GNU Guix|https://guix.gnu.org><index|Guix> can run
- <shell|guix shell -D gnunet-scheme> in the checkout to create an
- environment where these dependencies are all
- present.<space|1em>Scheme-GNUnet uses the standard GNU build system, so to
- build Scheme-Gnunet, you only need to run
-
- <\shell-code>
- autoreconf -vif
-
- ./configure
-
- make
-
- make check
- </shell-code>
-
- After building, the documentation is available at
- <verbatim|doc/scheme-gnunet.pdf> and <verbatim|doc/scheme-gnunet.html> in
- PDF and HTML formats.<space|1em>To get started, you can run the example
- mini-application at <verbatim|examples/nse-web.scm> and point your browser
- at <slink|http://localhost:8089>:
-
- <\shell-code>
- $ guile -L . -C . -l examples/nse-web.scm
- </shell-code>
-
- <subsection|Authenticating new source code><index|authentication>
-
- When GNU Guix is present, after pulling the latest Scheme-GNUnet commit,
- the following command can be run to verify it is authentic:
-
- <\shell-code>
- guix git authenticate 088ea8b27b95143584cbc36202c5a02dfe95796c "C1F3 3EE2
- 0C52 8FDB 7DD7 \ 011F 49E3 EE22 1917 25EE"
- </shell-code>
-
- If it isn't authentic, an error message such as the following will be
- written:
-
- <\shell-code>
- Authenticating commits 54a74dc to 431f336 (1 new commits)...
-
- [#########################################################]
-
- guix git: error: commit 431f336edd51e1f0fe059a6f6f2d4c3e9267b7bc not
- signed by an authorized key: C1F3 3EE2 0C52 8FDB 7DD7 \ 011F 49E3 EE22
- 1917 25EE
- </shell-code>
-
- <section|Writing tests>
-
- <index|tests>\<#2018\><hlink|How SQLite Is
- Tested|https://sqlite.org/testing.html>\<#2019\> is a recommended
- read.<space|1em>Scheme-GNUnet isn't that well-tested but still aims for
- being free of bugs and having many tests to prevents bugs from being
- introduced.<space|1em>When adding new code, consider writing test
- cases.<space|1em>Some things that can be tested and few methods for testing
- things:
-
- <\itemize>
- <item>Run mutation tests.<space|1em>That is, replace in the source code
- <scm|\<less\>> with <scm|\<less\>=>, <scm|0> with <scm|1>, a variable
- reference <scm|i> with a variable reference <scm|j>, swap destination and
- source arguments <text-dots> and verify whether the tests detect these
- little mutations.
-
- <item>Be exhaustive.<space|1em>If a procedure handles both foos and bars,
- write test cases that pass the procedure a foo and test cases that pass
- the procedure a bar.<space|1em>Sometimes Guile-QuickCheck can help with
- generating many test cases if the input has a regular structure yet many
- edge cases, see e.g. <verbatim|tests/cmsg.scm>.
-
- <item>Verify exception mechanisms!<space|1em>If a procedure is expected
- to handle I/O errors, simulate I/O errors and end-of-files in all the
- wrong places.<space|1em>If the procedure can raise exceptions, make sure
- these exceptions are raised when necessary.
- </itemize>
-
- Tests are added in the directory <scm|tests> and to the variable
- <verbatim|SCM_TESTS> in <verbatim|Makefile.am> and use <scm|srfi
- :64>.<space|1em>To run the test suite, run <verbatim|make check>.
-
- <section|Contact>
-
- Scheme-GNUnet is currently maintained on NotABug:
- <slink|https://notabug.org/maximed/scheme-gnunet/>.<space|1em>Issues and
- pull requests can be reported and submitted here.<space|1em>Alternatively,
- for discussion about developing Scheme-GNUnet, you can send mails to
- <hlink|gnunet-devel@gnu.org|mailto:gnunet-devel@gnu.org> and for help about
- how to use Scheme-GNUnet, you can contact
- <hlink|help-gnunet@gnu.org|mailto:help-gnunet@gnu.org>.<space|1em>These are
- public mailing lists, so don't send anything there you wouldn't mind the
- whole world to know.
-
- For security-sensitive issues, you can send a mail directly to the
- maintainer, <hlink|Maxime Devos
\<less\>maximedevos@telenet.be\<gtr\>|mailto:maximedevos@telenet.be>,
- optionally encrypted and signed with a GnuPG-compatible
- system.<space|1em>The maintainer's key fingerprint is C1F3 3EE2 0C52 8FDB
- 7DD7 011F 49E3 EE22 1917 25EE and a copy of the key can be downloaded from
- <slink|https://notabug.org/maximed/things/raw/master/Maxime_Devos.pub>.
-
- <section|License>
-
- The code of Scheme-GNUnet is available under the Affero General Public
- License (AGPL), version 3 or later; see individual source files for
- details.<space|1em>The documentaton is available under the GNU Free
- Documentation License, see the start of this manual and the likewise-named
- appendix for details.<space|1em>The AGPL has some unusual conditions w.r.t.
- applications interacting with the network, please read it carefully.
+ <include|contributing.tm>
<chapter|Application guide>
@@ -364,1014 +230,23 @@
<chapter|Configuration>
- There are a number of modules for accessing GNUnet
- configurations<index|configuration>.<space|1em>Firstly, there is <scm|(gnu
- gnunet config db)><index|(gnu gnunet config db)>, which is the module
- library code would typically use.<space|1em>For testing, one can create an
- empty configuration with the procedure
<scm|hash-\<gtr\>configuration><index|hash-\<gtr\>configuration>
- from that module and <scm|make-hashtable> from <scm|(rnrs hashtables)>,
- using <scm|hash-key><index|hash-key> as hash function and
- <scm|key=?><index|key=?> as comparison function:
-
- <\scm-code>
- (import (gnu gnunet config db)
-
- \ \ \ \ \ \ \ \ (gnu gnunet config value-parser)
-
- \ \ \ \ \ \ \ \ (rnrs hashtables))
-
- (define config (hash-\<gtr\>configuration (make-hashtable hash-key
- key=?))
- </scm-code>
-
- The resulting configuration <scm|config> is initially empty, so set some
- <with|font-shape|italic|keys> in the <with|font-shape|italic|section>
- <verbatim|nse>, to configure the network-size estimation service:
-
- <\scm-code>
- (set-value! identity config "nse" "UNIXPATH"
- "/tmp/nse.sock")<index|set-value!>
-
- (set-value! number-\<gtr\>string config "cadet" "MAX_ROUTES" 5000)
-
- ;; TODO: IP address, time durations, booleans, ...
- </scm-code>
-
- Now read these values back:
-
- <\scm-code>
- (read-value value-\<gtr\>file-name config "nse"
- "UNIXPATH")<index|read-value>
-
- ;; -\<gtr\> /tmp/nse.sock
-
- (read-value value-\<gtr\>natural config "cadet" "MAX_ROUTES")
-
- ;; -\<gtr\> 5000
- </scm-code>
-
- What if the configuration doesn't have a value for the specified section
- and key?<space|1em>Then an
<scm|&undefined-key-error><index|&undefined-key-error>
- results:
-
- <\scm-code>
- (read-value value-\<gtr\>natural config "kated" "MAX_ROUTES")
-
- ;; -\<gtr\>
-
- ;; ice-9/boot-9.scm:1685:16: In procedure raise-exception:
-
- ;; ERROR:
-
- ;; \ \ 1. &undefined-key-error:
-
- ;; \ \ \ \ \ \ section: "kated"
-
- ;; \ \ \ \ \ \ key: "MAX_ROUTES"
- </scm-code>
-
- <section|Locating configuration files>
-
- There are two \U possibly non-existent \U configuration files: the
- <with|font-shape|italic|user> configuration<subindex|configuration|user>
- and the <with|font-shape|italic|system>
- configuration<subindex|configuration|system>.<space|1em>The
- <with|font-shape|italic|system> configuration typically contains the paths
- for services like NSE, CORE, <text-dots><space|1em>A user may choose not to
- use the services by the system and instead use their own.<space|1em>To do
- so, the user needs to override the paths in the
- <with|font-shape|italic|user> configuration.<space|1em><todo|defaults?> The
- module <scm|(gnu gnunet config fs)><index|(gnu gnunet config fs)> is
- responsible for determining the location of the configuration files to load
- and actually load configuration files.<space|1em>For determining the
- location of the configuration files, the procedures
- <scm|locate-user-configuration> and <scm|locate-system-configuration> can
- be used.
-
- <\warning>
- The C implementation's mechanism for user-system separation seems to work
- differently.
- </warning>
-
- <\explain>
- <scm|(locate-user-configuration
<scm|#:getenv=getenv>)><index|locate-user-configuration>
- <|explain>
- This procedure determines the location of the user configuration file, as
- a string, or <scm|#false> if it could not be determined.<space|1em>If the
- location of the user configuration file is known, but the file does not
- exist, it is returned anyway, as a string.
-
- If the environment variable <shell|XDG_CONFIG_HOME> is set, the location
- of the file <verbatim|gnunet.conf> in the directory
- <shell|$XDG_CONFIG_HOME> is returned.<space|1em>If the environment
- variable is not set, the location of the file at
- <verbatim|.config/gnunet.conf><index|~/.config/gnunet.conf> in the home
- directory specified by the environment variable <verbatim|HOME> is
- returned, if that environment variable exists.<space|1em>If both are
- unset, <scm|#false> is returned.
-
- The values of environment variables is determined with the procedure
- <scm|getenv>.
- </explain>
-
- <\explain>
- <scm|(locate-system-configuration)><index|locate-system-configuration>
- <|explain>
- This procedure determines the location of the system configuration file,
- as a string.
-
- Currently, this is always
<verbatim|/etc/gnunet.conf><index|/etc/gnunet.conf>.
- </explain>
-
- <section|Loading configuration files>
-
- Once the location of the configuration file is known, the file can be
- opened with the Scheme procedure <scm|open-input-file>, which returns an
- input port.<space|1em>Then the procedure <scm|load-configuration/port!> can
- be used to determine all section-key-values triples in the configuration.
-
- <\explain>
- <scm|(load-configuration/port! <var|set-value!>
- <var|port>)><index|load-configuration/port!>
- <|explain>
- Load the configuration from the input port <var|port>.
-
- For each variable, call <var|set-value!> with the section name, variable
- name and a vector of the form <scm|#(line line-number value)>, where
- <var|value> a list of expansible objects.
-
- <todo|document expansible objects><todo|error reporting>
- </explain>
-
- A variable assignment<index|variable assignment> <verbatim|[section]
- key=value${var}> can refer to variables defined in the <verbatim|PATHS>
- section and variables from the environment.<space|1em>The previously
- described procedure <scm|load-configuration/port!> will <em|not> expand
- such assignements. \ To expand variable assignments, use the procedure
- <scm|make-expanded-configuration> instead.
-
- <\explain>
- <scm|(make-expanded-configuration <var|load!>
- #:getenv=<var|getenv>)><index|make-expanded-configuration>
- <|explain>
- Make a configuration object.<space|1em>To populate the configuration, all
- the procedure <var|load!> with a <scm|set-value!> procedure as expected
- by <scm|load-configuration/port!>.<space|1em>The values from
- <scm|set-value!> are added to the confoiguration and every variable is
- expanded.
- </explain>
-
- To automatically load the defaults, the system configuration and the user
- configuration, use the thunk <scm|load-configuration>:
-
- <\explain>
- <scm|(load-configuration #:getenv=<var|getenv>
- #:files=<text-dots>)><index|load-configuration>
- </explain|Load the defaults, the system configuration and the user
- configuration and return the resulting configuration object.<space|1em>The
- list of files to load can be overriden by setting the undocumented
- <var|files> keyword argument.>
-
- Applications (whether graphical or textual) are recommended to use
- <scm|load-configuration> by default, as it largely just works.
+ <include|configuration.tm>
<chapter|Manipulation of network structures>
- <index|network structure><index|netstruct>The modules <scm|(gnu gnunet
- netstruct procedural)> and <scm|(gnu gnunet netstruct syntactic)> can be
- used for formatting messages to be sent over the network or to a
- service.<space|1em>The macros <scm|define-type><index|define-type> and
- <scm|structure/packed><index|structure/packed> can be used to define new
- structures, like this:
-
- <\scm-code>
- (define-type /:msg:nse:estimate/example
-
- \ \ (structure/packed
-
- \ \ \ (properties '((message-symbol msg:nse:estimate)))
-
- \ \ \ (synopsis "Network size estimate")
-
- \ \ \ (documentation "Some explanation")
-
- \ \ \ (field (header /:message-header))
-
- \ \ \ (field (size-estimate ieee-double/big)
-
- \ \ \ \ \ \ \ \ \ \ (synopsis "Timestamp for when the estimate was
- made"))))
- </scm-code>
-
- This example is taken from the <scm|(gnu gnunet nse struct)> module and
- oversimplified.<space|1em>All its components will gradually
- explained.<space|1em>First, what actually is a network
- structure?<space|1em>This question is ambigious, because \<#2018\>network
- structure\<#2019\> can refer to either the <with|font-shape|italic|value>
- or the <with|font-shape|italic|type>.<space|1em>The
- <with|font-shape|italic|value> is a sequence of octets, i.e., a sequence of
- numbers in the closed range 0\U255.<space|1em>The
- <with|font-shape|italic|type> describes how the
- <with|font-shape|italic|value> is structured.
-
- As an example, consider figure <reference|networkstructex>.<space|1em>There,
- the value is <scm|0 12 1 65 64 51 238 123 71 27 58 149> and the type is
- <scm|/:msg:nse:estimate/example>.
-
- <big-figure|<tree|<scm|/:msg:nse:estimate/example>|<tree|<scm|header>
- <scm|/:message-header>|<tree|<scm|size> <scm|u16/big>|0
- 12>|<tree|<scm|type> <scm|u16/big>|1 65>>|<tree|<scm|size-estimate>
- <scm|ieee-double/big>|64 51 238 123 71 27 58 149>>|A network structure,
- both <with|font-shape|italic|value> and
- <with|font-shape|italic|type>.<label|networkstructex>>
-
- This value has a <with|font-shape|italic|header> with a
- <with|font-shape|italic|size> \<#2018\>0 0 0 12\<#2019\> of type
- <scm|u32/big>, so the <with|font-shape|italic|size> is 12.<space|1em>The
- <with|font-shape|italic|header> also has a <with|font-shape|italic|type>
- \<#2018\>0 0 1 65\<#2019\> of type <scm|u32/big>, so the type is
- <math|256\<times\>1+65=321>.<space|1em>The value also has a
- <with|font-shape|italic|size estimate> of type
- <scm|ieee-double/big>.<space|1em>The octets <scm|64 51 238 123 71 27 58
- 149>, \ interpreted as an IEEE double, form the number
- <math|19.93\<ldots\>>.
-
- <section|Documentation>
-
- A network structure can optionally have embedded
- documentation.<space|1em>More specifically, network structures can
- optionally have the <with|font-shape|italic|synopsis><index|synopsis>,
- <with|font-shape|italic|documentation><index|documentation> and
- <with|font-shape|italic|properties><index|properties> set.<space|1em>The
- <with|font-shape|italic|synopsis> is a short description of what the
- network structure represents, typically a one-liner.<space|1em>The
- <with|font-shape|italic|documentation> can be more detailed, explaining how
- the structure works, can be used, is used and should be used.<space|1em>The
- <with|font-shape|italic|properties> form a free-formed association list.
-
- The synopsis, documentation and properties can be set on structured created
- with <scm|structure/packed> and individual fields and can be accessed with
- the procedures <scm|documentation>, <scm|synopsis> and <scm|properties>.
-
- The following properties are defined in Scheme-GNUnet:
-
- <\description>
- <\item*>
- <\code>
- message-symbol
- </code>
- </item*>
-
- <index|message-symbol>For <with|font-shape|italic|message structures>,
- this is a list of the names of the <with|font-shape|italic|message types>
- (see <reference|sec:message type> Message type database) this structure
- can be used for \U most of the time, there's only a single such message
- type, but sometimes a single structure can be used for multiple message
- types.
-
- <item*|c-type>
-
- <label|c-type>The value is the name of the equivalent type in the C
- implementation, if any, as a symbol. This can be useful if you know the
- name of the C type but not the name of the Scheme type: in that case, you
- can do <shell|git grep -F GNUNET_MessageHeader> in a git checkout of the
- source code of Scheme-GNUnet to discover that Scheme-GNUnet's name for
- <cpp|GNUNET_MessageHeader> is <scm|/:message-header>.
- </description>
-
- <todo|TODO: it would be nice to use the properties and documentation to
- automatically create a form of documentation, with some cross-references to
- the C code>
-
- <section|Reading and writing>
-
- The procedures <scm|read%><index|read%>, <scm|set%!><index|set%!>,
- <scm|sizeof><index|sizeof> and <scm|select><index|select> from <scm|(gnu
- gnunet netstruct procedural)> or the like-named macros from <scm|(gnu
- gnunet netstruct syntactic)> can be used for reading and writing network
- structures.<space|1em>The macros from <scm|syntactic> behave like the
- procedures from <scm|procedural> with some optimisations at expansion
- time.<space|1em>The procedures will be demonstrated with the
- <scm|/:msg:nse:estimate/example> network structure defined previously.
-
- First, create a memory slice with <scm|make-slice/read-write> from
- <scm|(gnu gnunet utils bv-slice)>.<space|1em>The required size can be
- determinded with <scm|(sizeof /:msg:nse:estimate/example
- '())>.<space|1em>The role of <scm|'()> will be explained later.
-
- <\scm-code>
- (import (gnu gnunet netstruct procedural)
-
- \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic)
-
- \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)
-
- \ \ \ \ \ \ \ \ (gnu gnunet util struct))
-
- \;
-
- (define-type /:msg:nse:estimate/example
-
- \ \ (structure/packed
-
- \ \ \ (field (header /:message-header))
-
- \ \ \ (field (size-estimate ieee-double/big))))
-
- \;
-
- (define message
-
- \ \ (make-slice/read-write (sizeof /:msg:nse:estimate/example '())))
- </scm-code>
-
- The fields of <scm|message> can be set with <scm|(set%! netstruct '(field
- ...) slice value)>.<space|1em>The following code sets all the fields:
-
- <\scm-code>
- (set%! /:msg:nse:estimate/example '(header size) message
-
- \ \ \ \ \ \ \ (sizeof /:msg:nse:estimate/example '()))
-
- (set%! /:msg:nse:estimate/example '(header type) message 165)
-
- (set%! /:msg:nse:estimate/example '(size-estimate) message 19.2)
- </scm-code>
-
- The size of an individual field can be determined with <scm|(sizeof
- netstruct '(field ...))>.<space|1em>For example, the following code
- determines the size of the \<#2018\>size\<#2019\> field in the header:
-
- <\scm-code>
- (sizeof /:msg:nse:estimate/example '(header size)) ; 12
- </scm-code>
-
- The fields can also be read:
-
- <\scm-code>
- (read% /:msg:nse:estimate/example '(header size) message) ; 12
-
- (read% /:msg:nse:estimate/example '(header type) message) ; 165
-
- (read% /:msg:nse:estimate/example '(size-estimate) message) ; 19.2
- </scm-code>
-
- <section|Primitive types>
-
- There are a number of pre-defined types.<space|1em>First, there is
- <scm|u8>, a single octet that is interpreted as an integer in the closed
- range <math|<around*|[|0,255|]>>.<space|1em>There are also types
- <scm|uN/endian> for <math|N\<in\><around*|{|16,32,64|}>> and
- <math|endian\<in\><around*|{|little,big|}>>, which interprets <math|N/8>
- octets as integers in the closed range
<math|<around*|[|0,2<rsup|N>-1|]>>.<space|1em>The
- types <scm|ieee-double/big> and <scm|ieee-double/little> are 8 octets long
- and represent floating-point numbers in IEEE 754 format
- (\<#2018\>binary64\<#2019\>).
-
- <section|Packing>
-
- In contrast to C structures, Scheme-GNUnet network structures are always
- packed \V there are no \<#2018\>gaps\<#2019\> between fields.
+ <include|network-structures.tm>
<chapter|Communication with services>
- To connect with a GNUnet service<index|services> \V this applies to both
- the C and Scheme implementation, the GNUnet service must bind a local
- domain socket<\footnote>
- The C implementation supports Internet sockets as well.
- </footnote> somewhere on the file system and the client (possibly another
- service) must connect to it.<space|1em>Connections to a service can be made
- with the <scm|connect/fibers><index|connect/fibers> procedure from
- <scm|(gnu gnunet mq-impl stream)><index|(gnu gnunet mq-impl stream)>, like
- this:
-
- <\scm-code>
- (define mq (connect/fibers config "nse" handlers error-handler))
- </scm-code>
-
- <section|Asynchronuously connecting><index|connecting to services>
-
- This is an asynchronuous operation: it will \<#2018\>complete\<#2019\>
- immediately and the connection will actually be formed in the
- background.<space|1em>When the connection has actually be formed, the
- <scm|error-handler><index|error-handler> is called with the symbol
- <scm|connection:connected><index|connection:connected>.<space|1em>To
- demonstrate, the following code asynchronuously connects to the NSE
- service, and prints the text <scm|"connected!"> when the connection has
- actually been formed.
-
- <\scm-code>
- ;; XXX test this, explain 'config' ...
-
- (define (error-handler error . args)
-
- \ \ (case error
-
- \ \ \ \ ((connection:connected)
-
- \ \ \ \ \ (format #t "connected!~%"))
-
- \ \ \ \ (else (format #t "unknown error: ~a ~a~%" error args))))
-
- \;
-
- (define mq
-
- \ \ (connect/fibers config "nse" (message-handlers) error-handler))
- </scm-code>
-
- <section|Message handler>
-
- <index|message handler>When a message is received by the message queue, the
- corresponding message handler is invoked.<space|1em>Message handlers can be
- constructed with the <scm|message-handler><index|message-handler> macro and
- the <scm|make-message-handler><index|make-message-handler> procedure from
- <scm|(gnu gnunet mq handler)>, as follows:
-
- <\scm-code>
- (import (gnu gnunet mq handler)
-
- \ \ \ \ \ \ \ \ (gnu extractor enum)
-
- \ \ \ \ \ \ \ \ (gnu gnunet message protocols)
-
- \ \ \ \ \ \ \ \ (gnu gnunet util struct)
-
- \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)
-
- \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic))
-
- \;
-
- (define handler/syntactic
-
- \ \ (message-handler
-
- \ \ \ (type (symbol-value message-type msg:util:dummy))
-
- \ \ \ ((interpose code) code)
-
- \ \ \ ((well-formed? slice)
-
- \ \ \ \ (= (slice-length slice)
-
- \ \ \ \ \ \ \ (sizeof /:message-header '())))
-
- \ \ \ ((handle! slice)
-
- \ \ \ \ (pk 'message: slice))))
-
- \;
-
- (define handler/procedural
-
- \ \ (make-message-handler
-
- \ \ \ (symbol-value message-type msg:util:dummy)
-
- \ \ \ (lambda (thunk) (thunk))
-
- \ \ \ (lambda (slice)
-
- \ \ \ \ \ (= (slice-length slice)
-
- \ \ \ \ \ \ \ \ (sizeof /:message-header '())))
-
- \ \ \ (lambda (slice)
-
- \ \ \ \ \ (pk 'message: slice))))
- </scm-code>
-
- As illustrated in the example code above, a message handler has four
- components: the <with|font-shape|italic|type> of message<subindex|message
- type|of handler> it handles, an
<with|font-shape|italic|interposer><index|interposer>
- which will be explained later, the
<with|font-shape|italic|verifier><index|verifier>
- deciding if a message is well-formed and the
- <with|font-shape|italic|handler procedure><index|handler procedure>.
-
- The verifier is passed a bytevector slice with the message and should
- return <scm|#true> if the message is well-formed and <scm|#false> if it
- isn't.<space|1em>It may assume that the length of the slice corresponds to
- the length <em|in> the message header and is at least the length <em|of>
- the message header and that the type in the message header corresponds to
- the type of the message handler.<space|1em>Messages will only be passed to
- the handler procedue if the verifiers returns <scm|#true>.
-
- The handler procedure is passed a bytevector slice with the message, but
- only if the verifier considers it well-formed.<space|1em>The handler
- procedure and verifier are run from the
- <with|font-shape|italic|interposer>.<space|1em>The interposer is passed a
- thunk to execute and may e.g. install exception handlers and parameterise
- parameters.<space|1em>It can change the current input, output and error
- ports for example.
-
- <todo|document the message type database, various procedures>
-
- <section|Message type database><label|sec:message type><subindex|message
- type|database>
-
- The module <scm|(gnu gnunet message protocols)><index|(gnu gnunet message
- protocols)> has a mapping of symbolic names of every message type known to
- scheme-GNUnet to their numeric value.<space|1em>To use it, the macro
- <scm|symbol-value><index|symbol-value> from <scm|(gnu extractor
- enum)><index|(gnu extractor enum)> is required and possibly
- <scm|value-\<gtr\>index><index|value-\<gtr\>index> as well.<space|1em>To
- determine the numeric value of the message type <scm|msg:nse:estimate>, one
- would write:
-
- <\scm-code>
- (define numeric-type
-
- \ \ (value-\<gtr\>index (symbol-value message-type msg:nse:estimate)))
- </scm-code>
-
- <todo|other various enum procedures for introspection, documentation,
- <text-dots>?>
-
- <todo|how to define new message types>
-
- <section|Error handler><index|error handler>
-
- The message queue implementation usually just sends and receives messages,
- but some exceptional situations cannot be communicated with
- <scm|send-message!> or <scm|inject-message!>.<space|1em>For those, there is
- the <scm|inject-error!><index|inject-error!> procedure.<space|1em>This
- variadic procedure accepts a message queue to inject the error into, a
- <with|font-shape|italic|key><index|key> (usually a symbol) describing the
- exceptional situation and rest arguments.<space|1em>It calls the
- <with|font-shape|italic|error handler> of the message queue with the key
- and rest arguments.<space|1em>The following errors can currently be
- reported by the built-in message queue implementations:
-
- <\explain>
- <scm|connection:connected><index|connection:connected>
- <|explain>
- The connection to the server has been established.
- </explain>
-
- <\explain>
- <scm|connection:interrupted><index|connection:interrupted>
- </explain|The message queue has been closed before the connection to the
- server could be established.>
-
- <\explain>
- <scm|input:regular-end-of-file><index|input:regular-end-of-file>
- <|explain>
- The connection has been closed by the server.
-
- For people wondering about what happens if a connection becomes
- half-duplex: GNUnet does not have a notion of half-duplex message
- streams.<space|1em>If it is detected the underlying stream became
- half-duplex anyways, it will be treated as closed by scheme-GNUnet,
- resulting in this error.<space|1em>However, note that currently broken
- pipes cannot be reliably detected.
- </explain>
-
- <\explain>
- <scm|input:premature-end-of-file><index|input:premature-end-of-file>
- </explain|The connection was closed by the server while a message was still
- being read.<space|1em>This can happen if the server was stopped while it
- was still sending the rest of the message.>
-
- <\explain>
- <scm|input:overly-small> <var|type> <var|size><index|input:overly-small>
- </explain|The message size in the header was smaller than the minimal
- message size.<space|1em>Sometimes, but not always, the message type
- <var|type> and message size <var|size> are available (as exact
- naturals).<space|1em>When they are not available, <var|type> and <var|size>
- are <scm|#false> instead.<space|1em>This can only happen if the server or
- connection to the server is buggy.>
-
- <\explain>
- <scm|logic:no-handler> <var|type> . <var|rest><index|logic:no-handler>
- <|explain>
- The received message of type <var|type> (as an integer) does not have a
- corresponding message handler.<space|1em><var|rest> is currently
- unspecified.
- </explain>
-
- <\explain>
- <scm|logic:ill-formed> <var|type> . <var|rest><index|logic:ill-formed>
- </explain|The received message of type (as an integer) is ill-formed
- according to the message handler.<space|1em><var|rest> is currently
- unspecified.>
-
- Consider automatically reconnecting after
- <scm|<scm|input:regular-end-of-file>> and
- <scm|<scm|input:premature-end-of-file>>, to allow the server to restart
- without having to manually restart every individual
- application.<space|1em>To report errors, see the section
- <reference|sec:error reporting> Error reporting.
-
- <section|Ordering of injected errors and messages and sent messages>
-
- This section describes how injected errors and messages and sent messages
- are ordered with respect to each other in the default message queue
- implementation.<space|1em>Messages are handled or corresponding
- <scm|logic:no-handler> or <scm|logic:ill-formed> errors are injected in the
- order that the messages are received.<space|1em>Before messages are read,
- <scm|connection:connected> is injected.<space|1em>This error is injected at
- most once.
-
- <em|Soon> after all messages are read (and therefore
- <with|font-shape|italic|soon> after all handled messages or corresponding
- errors), the error <scm|input:regular-end-of-file>,
- <scm|input:overly-small> or <scm|input:premature-end-of-file> is
- injected.<space|1em>Only one of those errors can be injected for the entire
- lifetime of the message queue.
-
- Be aware that <em|soon> is not <em|immediate> here!<space|1em>For example,
- it is possible for a message to be received, the port closed, a message
- queued for sending, the closing of the port being detected by the write
- fiber, <scm|input:regular-end-of-file> being injected from the write fiber
- and the read fiber handling the received message, and the read fiber
- exiting because the port is closed, in that order.
-
- Messages are sent (and received on the other side) in the order they were
- enqueued for sending.<space|1em>Likewise, the notify-sent callback of
- enqueued messages are called in order.<space|1em>If the notify-sent
- callback is called, it is before the message is received by the other
- side.<space|1em>The message and its notify-sent callback are only received
- by the other side and called after the message has been injected and
- <scm|connection:connected> has been injected.<space|1em>It is possible for
- the notify-sent callback to be called without the message being received by
- the other side, e.g. if the port was closed during the notify-sent
- callback.
-
- If a message is received by the other side, all previously-sent messages
- have be received before.<space|1em>If a notify-sent callback is invoked,
- all notify-sent callbacks of previous messages have been invoked before,
- except the messages that are eventually cancelled.
-
- The errors <scm|logic:no-handler> and <scm|logic:ill-formed> are not fatal:
- later messages can still be read and handled.<space|1em>If
- <scm|connection:interrupted> is injected, no other errors are ever
- injected, whether in the past or in the future.<space|1em>This error can
- only be injected once.
-
- <todo|I/O errors>
-
- <todo|envelopes>
-
- <section|Disconnecting><index|disconnecting>
-
- A message queue can be closed with the <scm|close-queue!><index|close-queue!>
- procedure from <scm|(gnu gnunet mq)>.<space|1em>In the default message
- queue implementation, this asynchronuously closes the port and stops
- associated fibers.<space|1em>Closing ports when they won't be used anymore
- is important for limiting resource consumption, especially for servers that
- can have many connections.<space|1em>Closing message queues is an
- idempotent operation: closing a message queue twice is the same as closing
- it once.<space|1em> If a message queue is closed before a connection could
- be formed, <scm|connection:interrupted><index|connection:interrupted> is
- injected instead of <scm|connection:connected> and
- <scm|connection:regular-end-of-file>.
-
- <section|Error reporting><label|sec:error reporting>
-
- <index|error reporting>Errors can be reported with the procedure
- <scm|report-error> from the module <scm|(gnu gnunet mq
- error-reporting)><index|(gnu gnunet mq error-reporting)>.<space|1em>It can
- be called as <scm|(report-error key argument ...)>, e.g. <scm|(report-error
- 'logic:no-handler 3)><index|report-error>.<space|1em>By default, it reports
- the error to the current error port.<space|1em>If this is not desired, the
- output can be sent to another port by setting the parameter
-
<scm|textual-error-reporting-port><index|textual-error-reporting-port>.<space|1em>If
- textual error reporting is not desired, the parameter
- <scm|error-reporter><index|error-reporter> can be set to a procedure with
- the same interface as <scm|report-error>.<space|1em>Such a procedure could
- e.g. open a GUI dialog, sent the message to the system logger or ignore the
- error.
-
- Error messages are translated for the current locale.<todo|TODO actually
- call bindtextdomain>
+ <include|service-communication.tm>
<chapter|Estimation of the size of the network>
- <index|network size estimation><index|NSE>GNUnet has a service that roughly
- estimates the size of the network \U i.e., the number of
- peers.<space|1em>The module <scm|(gnu gnunet nse client)><index|(gnu gnunet
- nse client)> can be used to interact with this service.<space|1em>The
- connection is made with the procedure <scm|connect><subindex|connect|NSE>,
- which is accepts a <with|font-shape|italic|configuration> (see
- <todo|reference>) and some optional keyword arguments.<space|1em>This
- procedure can be called as <scm|(connect config #:updated updated
- #:connected connected #:disconnected disconnected)>.<space|1em>It returns a
- <with|font-shape|italic|NSE server object><index|NSE server
- object><subindex|server object|NSE>.
-
- The connection is made asynchronuously; the thunk <var|connected> will be
- called when the connection has actually been made.<space|1em>Whenever a new
- estimate becomes available, the (optional) procedure
- <var|updated><index|update procedure> is called with the new
- <with|font-shape|italic|estimate><index|estimate
- object>.<space|1em>Alternatively, the procedure
- <scm|estimate><index|estimate> can be called on the server object to return
- the latest available estimate.<space|1em>If the
- <with|font-shape|italic|server object> doesn't have an estimate yet, that
- procedure will return <scm|#false> instead of an estimate.
-
- When the connection is lost, the (optional) thunk <var|disconnected> is
- called and <scm|(gnu gnunet nse client)> will retry
- connecting.<space|1em>To close the current connection, if any, and stop
- reconnecting, the idempotent procedure
<scm|disconnect!><subindex|disconnect!|NSE>
- can be called on the server object.
-
- <todo|input, validation, I/O errors?>
-
- The estimate object has a number of accessors:
-
- <\explain>
- <scm|(estimate:logarithmic-number-peers
- <var|estimate>)><index|estimate:logarithmic-number-peers>
- </explain|The base-2 logarithm of the number of peers (estimated), as a
- positive flonum, possibly zero or infinite>
-
- <\explain>
- <scm|(estimate:number-peers <var|estimate>)><index|estimate:number-peeers>
- </explain|The number of peers (estimated), as a flonum, at least <scm|1.0>
- and possibly infinite.<space|1em>This is not necessarily an (inexact)
- <scm|integer?>, as it is only an estimate.>
-
- <\explain>
- <scm|(estimate:timestamp estimate)><index|estimate:timestamp>
- </explain|A timestamp when the estimate was made <todo|something about
- epoch?>>
-
- <\explain>
- <scm|(estimate:standard-deviation
<var|estimate>)><index|estimate:standard-deviation>
- <|explain>
- The estimated standard deviation on the base-2 logarithm of peers,
- calculated over the last 64 rounds, with the <math|<frac|N|N-1>>
- correction.<space|1em>This is a positive flonum, possibly zero or
- infinite, or not-a-number (indicating division of zero by zero).
-
- If the peer has been connected to the network for a while, the number is
- expected to be finite and strictly positive.
- </explain>
-
- Assuming the network size is stable and the errors on the logarithmic
- estimate are normally distributed, the procedure
- <scm|estimate:standard-deviation> can be used to put probablistic error
- bounds on the number of peers on the network. <todo|example>
+ <include|network-size-estimation.tm>
<chapter|Accessing the DHT>
- GNUnet has a service that maintains a <em|distributed hash
- table><index|distributed hash table><index|DHT><index|R5N>, mapping keys to
- values. The module <scm|(gnu gnunet dht client)><index|(gnu gnunet dht
- client)> can be used to interact with the service. The connection can be
- made with the procedure <scm|connect>. It returns a <em|DHT server
- object><index|DHT server object><subindex|server object|DHT>.
-
- <\explain>
- <scm|(connect <var|config> <var|#:connected> <var|#:disconnected>
- <var|#:spawn>)><subindex|connect|DHT>
- <|explain>
- Connect to the DHT service, using the configuration <var|config>. The
- connection is made asynchronuously; the optional thunk <var|connected> is
- called when the connection has been made. The connection can break; the
- optional thunk <var|disconnected> is called when it does. If the
- connection breaks, the client code automatically tries to reconnect, so
- <var|connected> can be called after <var|disconnected>.
- </explain>
-
- <\explain>
- <scm|(disconnect! <var|server>)><subindex|disconnect!|DHT>
- <|explain>
- Asynchronuously disconnect from the DHT service and stop reconnecting,
- even if not connected. This is an idempotent operation.
- </explain>
-
- <section|Data in the DHT>
-
- To insert data into the DHT, the DHT service needs various information \U
- the key and the value, but also some other information. Likewise, when the
- DHT service has found the datum, this information is available as well. A
- datum in the DHT is represented as a <em|datum object><index|datum object>.
- The object holding the information required for inserting something in the
- DHT is called an <em|insertion object><index|insertion object>. Likewise,
- the information of searching for something in the DHT is in a <em|query
- object><index|query object>. The result of a query is a <em|search result
- object><index|search result object>.
-
- These objects only hold information; creating them does not have any side
- effects.
-
- <\explain>
- <scm|(make-datum <var|type> <var|key> <var|value>
- <var|#:expiration>)><index|make-datum>
- <|explain>
- Make a datum object of block type <var|type> (or its corresponding
- numeric value), with key <var|key> (a readable <scm|/hashcode:512>
- bytevector slice), value <var|value> (a readable bytevector slice) and
- expiring at <var|expiration> (<todo|type, epoch>). The keyword argument
- <var|expiration> is optional, see <reference|???>.
-
- The numeric value of the block type can be retrieved with the accessor
- <scm|datum-type>. The accessors <scm|datum-key><index|datum-key>,
- <scm|datum-value><index|datum-value> and
- <scm|datum-expiration><index|datum-expiration> return the key, value and
- expiration time respectively. It can be tested if an object is a datum
- object with the predicate <scm|datum?><index|datum?>.
-
- The length of <var|value> may be at most
- <scm|%max-datum-value-length><index|%max-datum-value-length>. If this
- bound is exceeded, an appropriate
<scm|&overly-large-datum><index|&overly-large-datum>
- and <scm|&who> condition is raised.
- </explain>
-
- <\explain>
- <scm|(datum-\<gtr\>insertion <var|datum>
- #:desired-replication-level)><index|datum-\<gtr\>insertion>
- <|explain>
- Make an insertion object for inserting the datum <var|datum>, desiring a
- replication level <var|desired-replication-level> (see
- <reference|replication levels???>)<todo|various options>.
-
- The datum and desired replication level can be recovered with the
- accessors <scm|insertion-\<gtr\>datum><index|insertion-\<gtr\>datum> and
-
<var|insertion-desired-replication-level><index|insertion-desired-replication-level>.
- It can be tested if an object is an insertion object with the predicate
- <scm|insertion?><index|insertion?>.
- </explain>
-
- <\explain>
- <scm|(make-query <var|type> <var|key>
- #:desired-replication-level)><index|make-query>
- <|explain>
- Make a query object for searching for a value of block type <var|type>
- (or its corresponding numeric value), with key <var|key> (a readable
- <scm|/hashcode:512> bytevector slice), at desired replication level
- <scm|desired-replication-level> (see <reference|replication levels???>).
- <todo|various options, xquery>
-
- The numeric value of the block type, the key and the desired replication
- level can be recovered with the accessors
- <scm|query-type><index|query-type>, <scm|query-key><index|query-key> and
-
<scm|query-desired-replication-level><index|query-desired-replication-level>.
- It can be tested if an object is a query object with the predicate
- <scm|query?><index|query?>.
- </explain>
-
- <\explain>
- <scm|(datum-\<gtr\>search-result <var|datum> #:get-path
- #:put-path)><index|datum-\<gtr\>search-result>
- <|explain>
- Make a search result object for the datum <var|datum>. The datum can be
- recovered with the accessor
<scm|search-result-\<gtr\>datum><index|search-result-\<gtr\>datum>.
- It can be tested if an object is a search result with the predicate
- <scm|search-result?><index|search-result?>. The optional arguments
- <var|get-path> and <var|put-path>, when not false, are bytevector slices
- consisting of a list of
<scm|/dht:path-element><index|/dht:path-element><index|path
- element>.
-
- The <var|get-path><index|get path> , if any, is the path from the storage
- location to the current peer. Conversely, the <var|put-path><index|put
- path>, if any, is a path from the peer that inserted the datum into the
- DHT to the storage location. The <var|get-path> and <var|put-path> can be
- accessed with <scm|search-result-get-path><index|search-result-get-path>
- and <scm|search-result-put-path><index|search-result-put-path>
- respectively.
-
- When the datum, get path and put path together are too large, a
- <scm|&overly-large-paths><index|&overly-large-paths> condition is raised.
- When the bytevector slice length of <var|get-path> or <var|put-path> is
- not a multiple of the size of a path element, then a
- <scm|&malformed-path><index|&malformed-path> condition is raised.
- </explain>
-
- <section|Accessing data in the DHT>
-
- To insert a datum into the DHT, the procedure <scm|put!> is used. To find
- data matching a query, the procedure <scm|start-get!> is
- used.<index|searching the DHT><index|inserting data into the DHT>
-
- <\explain>
- <scm|(start-get! <var|server> <var|query> <var|found>)><index|start-get!>
- <|explain>
- Search for data matching <var|query> in the DHT. When a datum is found,
- call the unary procedure <var|found> on the search result. It is possible
- to find multiple data matching a query. In that case, <var|found> is
- called multiple times. Searching happens asynchronuously; to stop the
- search, a fresh <em|search object><index|search object> for controlling
- the search is returned.
-
- The procedure <var|found> is run from the context of <var|server>. As
- such, if <var|found> blocks, then all operations on <var|server> might
- block. As such, it is recommended for <var|found> to do as little as
- possible by itself and instead delegate any work to a separate fiber.
-
- To avoid expensive copies, the implementation can choose to reuse
- internal buffers for the slices passed to <var|found>, which could be
- overwritten after the call to <var|found>. As such, it might be necessary
- to make a copy of the search result, using <scm|copy-search-result>.
- </explain>
-
- <\explain>
- <scm|(put! <var|server> <var|insertion> <var|#:confirmed>)><index|put!>
- <|explain>
- Perform the insertion <var|insertion>. When the datum has been inserted,
- the optional thunk <var|confirmed> is called. A <em|put object> is
- returned which can be used to cancel the insertion.
-
- <todo|TODO: actually call <var|confirmed>>
- </explain>
-
- <\explain>
- <scm|(copy-query <var|old>)><index|copy-query>
-
- <scm|(copy-datum <var|old>)><index|copy-datum>
-
- <scm|(copy-insertion <var|old>)><index|copy-insertion>
-
- <scm|(copy-search-result <var|old>)><index|copy-search-result>
- <|explain>
- Make a copy of the object <var|old> (a query, datum, insertion or search
- result object, depending on the procedure), such that modifications to
- the slices in <var|old> do not impact the new object.
- </explain>
-
- <todo|cancellation>
-
- <section|Constructing and analysing network messages>
-
- The DHT client and service communicate by sending <em|messages>. Usually,
- only the implementation of the client and service need to construct and
- analyse these messages, but nothing prevents other uses of the procedures
- in <scm|(gnu gnunet dht network)><index|(gnu gnunet dht network)>, e.g. for
- learning, in a tool like Wireshark or for tests.
-
- The <em|analysis> procedures<index|analysis procedures> assume that the
- message is well-formed and avoid constructing new bytevector slices by
- taking subslices. The <em|construction> procedures<index|construction
- procedures> create fresh well-formed read-write bytevector slices.
-
- <\warning>
- Possibly the type of <var|options> will change and possibly the options
- will be moved into the query object and insertion object.
- </warning>
-
- <\explain>
- <scm|(construct-client-get <var|query> <var|unique-id> #:optional
- (<var|options> 0))><index|construct-client-get>
- <|explain>
- Create a new <scm|/:msg:dht:client:get><index|/:msg:dht:client:get>
- message for the query object <var|query>, with <var|unique-id> as
- \<#2018\>unique id\<#2019\> and <var|options> as options.
- </explain>
-
- <\explain>
- <scm|(construct-client-put <var|insertion> #:optional (options
- 0))><index|construct-client-put>
- <|explain>
- Create a new <scm|/:msg:dht:client:put><index|/:msg:dht:client:put>
- message for the insertion object <var|insertion> with <var|options> as
- options.
- </explain>
-
- <\explain>
- <scm|(construct-client-result <var|search-result>
- <var|unique-id>)><index|construct-client-result>
- <|explain>
- Create a new <scm|/:msg:dht:client:result><index|/:msg:dht:client:result>
- message for the search result object <var|search-result>, with
- <var|unique-id> as \<#2018\>unique id\<#2019\> .
- </explain>
-
- <\explain>
- <scm|(analyse-client-get <var|message>)><index|analyse-client-get>
- <|explain>
- Return the query object, the unique id and the options corresponding to
- the <scm|/:msg:dht:client:result><index|/:msg:dht:client:result> message
- <var|message>. Xqueries are currently unsupported.
- </explain>
-
- <\explain>
- <scm|(analyse-client-put <var|message>)><index|analyse-client-put>
- <|explain>
- Return the insertion object and options corresponding to the
- <scm|/:msg:dht:client:put><index|/:msg:dht:client:put> message
- <var|message>.
- </explain>
-
- <\explain>
- <scm|(analyse-client-result <var|message>)><index|analyse-client-result>
- <|explain>
- Return search result object and unique id for the
- <scm|/:msg:dht:client:result><index|/:msg:dht:client:result> message
- <var|message>.
- </explain>
-
- <todo|monitoring messages>
-
- <section|How to handle invalid data>
-
- <todo|todo!>
-
- <section|Monitoring: spying on what other applications and peers are doing>
-
- <todo|todo!>
+ <include|distributed-hash-table.tm>
<chapter|Cryptography>
@@ -1699,6 +574,7 @@
<\initial>
<\collection>
<associate|page-medium|paper>
+ <associate|project-flag|true>
<associate|save-aux|false>
</collection>
</initial>
\ No newline at end of file
diff --git a/doc/service-communication.tm b/doc/service-communication.tm
new file mode 100644
index 0000000..056be14
--- /dev/null
+++ b/doc/service-communication.tm
@@ -0,0 +1,325 @@
+<TeXmacs|2.1>
+
+<project|scheme-gnunet.tm>
+
+<style|tmmanual>
+
+<\body>
+ To connect with a GNUnet service<index|services> \V this applies to both
+ the C and Scheme implementation, the GNUnet service must bind a local
+ domain socket<\footnote>
+ The C implementation supports Internet sockets as well.
+ </footnote> somewhere on the file system and the client (possibly another
+ service) must connect to it.<space|1em>Connections to a service can be made
+ with the <scm|connect/fibers><index|connect/fibers> procedure from
+ <scm|(gnu gnunet mq-impl stream)><index|(gnu gnunet mq-impl stream)>, like
+ this:
+
+ <\scm-code>
+ (define mq (connect/fibers config "nse" handlers error-handler))
+ </scm-code>
+
+ <section|Asynchronuously connecting><index|connecting to services>
+
+ This is an asynchronuous operation: it will \<#2018\>complete\<#2019\>
+ immediately and the connection will actually be formed in the
+ background.<space|1em>When the connection has actually be formed, the
+ <scm|error-handler><index|error-handler> is called with the symbol
+ <scm|connection:connected><index|connection:connected>.<space|1em>To
+ demonstrate, the following code asynchronuously connects to the NSE
+ service, and prints the text <scm|"connected!"> when the connection has
+ actually been formed.
+
+ <\scm-code>
+ ;; XXX test this, explain 'config' ...
+
+ (define (error-handler error . args)
+
+ \ \ (case error
+
+ \ \ \ \ ((connection:connected)
+
+ \ \ \ \ \ (format #t "connected!~%"))
+
+ \ \ \ \ (else (format #t "unknown error: ~a ~a~%" error args))))
+
+ \;
+
+ (define mq
+
+ \ \ (connect/fibers config "nse" (message-handlers) error-handler))
+ </scm-code>
+
+ <section|Message handler>
+
+ <index|message handler>When a message is received by the message queue, the
+ corresponding message handler is invoked.<space|1em>Message handlers can be
+ constructed with the <scm|message-handler><index|message-handler> macro and
+ the <scm|make-message-handler><index|make-message-handler> procedure from
+ <scm|(gnu gnunet mq handler)>, as follows:
+
+ <\scm-code>
+ (import (gnu gnunet mq handler)
+
+ \ \ \ \ \ \ \ \ (gnu extractor enum)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet message protocols)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet util struct)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet utils bv-slice)
+
+ \ \ \ \ \ \ \ \ (gnu gnunet netstruct syntactic))
+
+ \;
+
+ (define handler/syntactic
+
+ \ \ (message-handler
+
+ \ \ \ (type (symbol-value message-type msg:util:dummy))
+
+ \ \ \ ((interpose code) code)
+
+ \ \ \ ((well-formed? slice)
+
+ \ \ \ \ (= (slice-length slice)
+
+ \ \ \ \ \ \ \ (sizeof /:message-header '())))
+
+ \ \ \ ((handle! slice)
+
+ \ \ \ \ (pk 'message: slice))))
+
+ \;
+
+ (define handler/procedural
+
+ \ \ (make-message-handler
+
+ \ \ \ (symbol-value message-type msg:util:dummy)
+
+ \ \ \ (lambda (thunk) (thunk))
+
+ \ \ \ (lambda (slice)
+
+ \ \ \ \ \ (= (slice-length slice)
+
+ \ \ \ \ \ \ \ \ (sizeof /:message-header '())))
+
+ \ \ \ (lambda (slice)
+
+ \ \ \ \ \ (pk 'message: slice))))
+ </scm-code>
+
+ As illustrated in the example code above, a message handler has four
+ components: the <with|font-shape|italic|type> of message<subindex|message
+ type|of handler> it handles, an
<with|font-shape|italic|interposer><index|interposer>
+ which will be explained later, the
<with|font-shape|italic|verifier><index|verifier>
+ deciding if a message is well-formed and the
+ <with|font-shape|italic|handler procedure><index|handler procedure>.
+
+ The verifier is passed a bytevector slice with the message and should
+ return <scm|#true> if the message is well-formed and <scm|#false> if it
+ isn't.<space|1em>It may assume that the length of the slice corresponds to
+ the length <em|in> the message header and is at least the length <em|of>
+ the message header and that the type in the message header corresponds to
+ the type of the message handler.<space|1em>Messages will only be passed to
+ the handler procedue if the verifiers returns <scm|#true>.
+
+ The handler procedure is passed a bytevector slice with the message, but
+ only if the verifier considers it well-formed.<space|1em>The handler
+ procedure and verifier are run from the
+ <with|font-shape|italic|interposer>.<space|1em>The interposer is passed a
+ thunk to execute and may e.g. install exception handlers and parameterise
+ parameters.<space|1em>It can change the current input, output and error
+ ports for example.
+
+ <todo|document the message type database, various procedures>
+
+ <section|Message type database><label|sec:message type><subindex|message
+ type|database>
+
+ The module <scm|(gnu gnunet message protocols)><index|(gnu gnunet message
+ protocols)> has a mapping of symbolic names of every message type known to
+ scheme-GNUnet to their numeric value.<space|1em>To use it, the macro
+ <scm|symbol-value><index|symbol-value> from <scm|(gnu extractor
+ enum)><index|(gnu extractor enum)> is required and possibly
+ <scm|value-\<gtr\>index><index|value-\<gtr\>index> as well.<space|1em>To
+ determine the numeric value of the message type <scm|msg:nse:estimate>, one
+ would write:
+
+ <\scm-code>
+ (define numeric-type
+
+ \ \ (value-\<gtr\>index (symbol-value message-type msg:nse:estimate)))
+ </scm-code>
+
+ <todo|other various enum procedures for introspection, documentation,
+ <text-dots>?>
+
+ <todo|how to define new message types>
+
+ <section|Error handler><index|error handler>
+
+ The message queue implementation usually just sends and receives messages,
+ but some exceptional situations cannot be communicated with
+ <scm|send-message!> or <scm|inject-message!>.<space|1em>For those, there is
+ the <scm|inject-error!><index|inject-error!> procedure.<space|1em>This
+ variadic procedure accepts a message queue to inject the error into, a
+ <with|font-shape|italic|key><index|key> (usually a symbol) describing the
+ exceptional situation and rest arguments.<space|1em>It calls the
+ <with|font-shape|italic|error handler> of the message queue with the key
+ and rest arguments.<space|1em>The following errors can currently be
+ reported by the built-in message queue implementations:
+
+ <\explain>
+ <scm|connection:connected><index|connection:connected>
+ <|explain>
+ The connection to the server has been established.
+ </explain>
+
+ <\explain>
+ <scm|connection:interrupted><index|connection:interrupted>
+ </explain|The message queue has been closed before the connection to the
+ server could be established.>
+
+ <\explain>
+ <scm|input:regular-end-of-file><index|input:regular-end-of-file>
+ <|explain>
+ The connection has been closed by the server.
+
+ For people wondering about what happens if a connection becomes
+ half-duplex: GNUnet does not have a notion of half-duplex message
+ streams.<space|1em>If it is detected the underlying stream became
+ half-duplex anyways, it will be treated as closed by scheme-GNUnet,
+ resulting in this error.<space|1em>However, note that currently broken
+ pipes cannot be reliably detected.
+ </explain>
+
+ <\explain>
+ <scm|input:premature-end-of-file><index|input:premature-end-of-file>
+ </explain|The connection was closed by the server while a message was still
+ being read.<space|1em>This can happen if the server was stopped while it
+ was still sending the rest of the message.>
+
+ <\explain>
+ <scm|input:overly-small> <var|type> <var|size><index|input:overly-small>
+ </explain|The message size in the header was smaller than the minimal
+ message size.<space|1em>Sometimes, but not always, the message type
+ <var|type> and message size <var|size> are available (as exact
+ naturals).<space|1em>When they are not available, <var|type> and <var|size>
+ are <scm|#false> instead.<space|1em>This can only happen if the server or
+ connection to the server is buggy.>
+
+ <\explain>
+ <scm|logic:no-handler> <var|type> . <var|rest><index|logic:no-handler>
+ <|explain>
+ The received message of type <var|type> (as an integer) does not have a
+ corresponding message handler.<space|1em><var|rest> is currently
+ unspecified.
+ </explain>
+
+ <\explain>
+ <scm|logic:ill-formed> <var|type> . <var|rest><index|logic:ill-formed>
+ </explain|The received message of type (as an integer) is ill-formed
+ according to the message handler.<space|1em><var|rest> is currently
+ unspecified.>
+
+ Consider automatically reconnecting after
+ <scm|<scm|input:regular-end-of-file>> and
+ <scm|<scm|input:premature-end-of-file>>, to allow the server to restart
+ without having to manually restart every individual
+ application.<space|1em>To report errors, see the section
+ <reference|sec:error reporting> Error reporting.
+
+ <section|Ordering of injected errors and messages and sent messages>
+
+ This section describes how injected errors and messages and sent messages
+ are ordered with respect to each other in the default message queue
+ implementation.<space|1em>Messages are handled or corresponding
+ <scm|logic:no-handler> or <scm|logic:ill-formed> errors are injected in the
+ order that the messages are received.<space|1em>Before messages are read,
+ <scm|connection:connected> is injected.<space|1em>This error is injected at
+ most once.
+
+ <em|Soon> after all messages are read (and therefore
+ <with|font-shape|italic|soon> after all handled messages or corresponding
+ errors), the error <scm|input:regular-end-of-file>,
+ <scm|input:overly-small> or <scm|input:premature-end-of-file> is
+ injected.<space|1em>Only one of those errors can be injected for the entire
+ lifetime of the message queue.
+
+ Be aware that <em|soon> is not <em|immediate> here!<space|1em>For example,
+ it is possible for a message to be received, the port closed, a message
+ queued for sending, the closing of the port being detected by the write
+ fiber, <scm|input:regular-end-of-file> being injected from the write fiber
+ and the read fiber handling the received message, and the read fiber
+ exiting because the port is closed, in that order.
+
+ Messages are sent (and received on the other side) in the order they were
+ enqueued for sending.<space|1em>Likewise, the notify-sent callback of
+ enqueued messages are called in order.<space|1em>If the notify-sent
+ callback is called, it is before the message is received by the other
+ side.<space|1em>The message and its notify-sent callback are only received
+ by the other side and called after the message has been injected and
+ <scm|connection:connected> has been injected.<space|1em>It is possible for
+ the notify-sent callback to be called without the message being received by
+ the other side, e.g. if the port was closed during the notify-sent
+ callback.
+
+ If a message is received by the other side, all previously-sent messages
+ have be received before.<space|1em>If a notify-sent callback is invoked,
+ all notify-sent callbacks of previous messages have been invoked before,
+ except the messages that are eventually cancelled.
+
+ The errors <scm|logic:no-handler> and <scm|logic:ill-formed> are not fatal:
+ later messages can still be read and handled.<space|1em>If
+ <scm|connection:interrupted> is injected, no other errors are ever
+ injected, whether in the past or in the future.<space|1em>This error can
+ only be injected once.
+
+ <todo|I/O errors>
+
+ <todo|envelopes>
+
+ <section|Disconnecting><index|disconnecting>
+
+ A message queue can be closed with the <scm|close-queue!><index|close-queue!>
+ procedure from <scm|(gnu gnunet mq)>.<space|1em>In the default message
+ queue implementation, this asynchronuously closes the port and stops
+ associated fibers.<space|1em>Closing ports when they won't be used anymore
+ is important for limiting resource consumption, especially for servers that
+ can have many connections.<space|1em>Closing message queues is an
+ idempotent operation: closing a message queue twice is the same as closing
+ it once.<space|1em> If a message queue is closed before a connection could
+ be formed, <scm|connection:interrupted><index|connection:interrupted> is
+ injected instead of <scm|connection:connected> and
+ <scm|connection:regular-end-of-file>.
+
+ <section|Error reporting><label|sec:error reporting>
+
+ <index|error reporting>Errors can be reported with the procedure
+ <scm|report-error> from the module <scm|(gnu gnunet mq
+ error-reporting)><index|(gnu gnunet mq error-reporting)>.<space|1em>It can
+ be called as <scm|(report-error key argument ...)>, e.g. <scm|(report-error
+ 'logic:no-handler 3)><index|report-error>.<space|1em>By default, it reports
+ the error to the current error port.<space|1em>If this is not desired, the
+ output can be sent to another port by setting the parameter
+
<scm|textual-error-reporting-port><index|textual-error-reporting-port>.<space|1em>If
+ textual error reporting is not desired, the parameter
+ <scm|error-reporter><index|error-reporter> can be set to a procedure with
+ the same interface as <scm|report-error>.<space|1em>Such a procedure could
+ e.g. open a GUI dialog, sent the message to the system logger or ignore the
+ error.
+
+ Error messages are translated for the current locale.<todo|TODO actually
+ call bindtextdomain>
+</body>
+
+<\initial>
+ <\collection>
+ <associate|page-medium|paper>
+ <associate|save-aux|false>
+ </collection>
+</initial>
\ No newline at end of file
--
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.
- [gnunet-scheme] branch master updated (8814f16 -> eda9192), gnunet, 2022/02/06
- [gnunet-scheme] 02/07: Define dependency information for protocols.go., gnunet, 2022/02/06
- [gnunet-scheme] 01/07: Define CADET message types., gnunet, 2022/02/06
- [gnunet-scheme] 05/07: Eliminate guix.scm., gnunet, 2022/02/06
- [gnunet-scheme] 04/07: cadet: Define some CADET network structures., gnunet, 2022/02/06
- [gnunet-scheme] 03/07: Correct license (GPL->AGPL) in the scmfrag., gnunet, 2022/02/06
- [gnunet-scheme] 07/07: Remove now unused patches., gnunet, 2022/02/06
- [gnunet-scheme] 06/07: doc: Split relatively large chapters into separate files.,
gnunet <=