# # # patch "Makefile.am" # from [469045bc84ad30c82fe8c8bfb5f98841d0ccc507] # to [21206f911138da0beabc2121becb8d7392c064e3] # # patch "automate.cc" # from [a0ff144fedc680631087a881a1876e9bf606d309] # to [bf0bf00deee9db8789f89f44d7d90fcb53e82179] # # patch "cmd_db.cc" # from [0875bc8882c861722b83089736d55d53469fa0c7] # to [45ddd81bad639a7a1d2f835c0c9d6a2d29b71cd0] # # patch "cmd_key_cert.cc" # from [a411628241629ecb7dcfbc490502701b62a63321] # to [e7798f640fc449740946bbc15f3a988f7570e344] # # patch "cmd_merging.cc" # from [ed3530c95d77f9ec3d2f9a1fdcfabca9bc05e10c] # to [1ee002f467d430ba43200e7f614cdf7e571049f5] # # patch "cmd_netsync.cc" # from [689f594c47cfc766e65df0e93159ec1f121ff800] # to [5f30be53c96420d71a67c75e98897c1532059b8c] # # patch "cmd_othervcs.cc" # from [6894f2644573582dadfeccb713ae3362c2953189] # to [ba717ebc34573dc41258fe4f679632876097eb48] # # patch "cmd_ws_commit.cc" # from [729a6ead62f91c4f1c8629938f15b5cfba69c5f4] # to [fc8e63fddea457889a603521c09cd199fb997bef] # # patch "globish.cc" # from [338d0214a12f6adaf4bda966e495fefb91e3a054] # to [1a7b2b42320d0c8d8fde56b5af871bc125d17a5a] # # patch "globish.hh" # from [864db1e5341486754cbb90b73eca6e4e0886f1d6] # to [93fca3ce5259a69e5ad8020c6d51560a696de54b] # # patch "keys.cc" # from [e27fd43f8730dc5ada1f4aaddfb55e910ac659ac] # to [49e561d782572c02d86f9592e640d67b46cb9948] # # patch "keys.hh" # from [6b5412008a7158b8abfb8228163b687538d7879f] # to [8b5f0676693fa00e68f7132e50abbc22b5097252] # # patch "netcmd.hh" # from [f6c60950810184737024220f58f806db5ec1ee2a] # to [6ac2e5bcf92deae29cbad403b4a0397f0a882734] # # patch "netsync.cc" # from [dd63fc06d6bab3dd3e8c30a1df6c23975bf94a54] # to [7d2cf5774d36b4fca9c1434f54f60a424f39f7db] # # patch "network/connection_info.cc" # from [f1426c49f3d6d2a8492c444f2555f5de5de75a1d] # to [ab2085f8d0836e1234588f605ff37679f3ae45bd] # # patch "network/connection_info.hh" # from [b1cd70d21e53940d512665deea9db7df53207951] # to [4a6bf8a4ce7bcb2519ad1a993a8ab527e2ab92c7] # # patch "options_list.hh" # from [8ce5a0d99837f816210671753b564550a563183b] # to [8a5da3dc9c861616ba52de164348285ea902e432] # ============================================================ --- Makefile.am 469045bc84ad30c82fe8c8bfb5f98841d0ccc507 +++ Makefile.am 21206f911138da0beabc2121becb8d7392c064e3 @@ -63,7 +63,7 @@ MOST_SOURCES = \ migration.hh migrate_schema.cc migrate_ancestry.cc \ refiner.cc refiner.hh \ enumerator.cc enumerator.hh \ - netsync.cc \ + netsync.hh netsync.cc \ network/automate_session.hh network/automate_session.cc \ network/connection_info.hh network/connection_info.cc \ network/listener_base.hh network/listener_base.cc \ ============================================================ --- automate.cc a0ff144fedc680631087a881a1876e9bf606d309 +++ automate.cc bf0bf00deee9db8789f89f44d7d90fcb53e82179 @@ -2114,7 +2114,7 @@ CMD_AUTOMATE(cert, N_("REVISION-ID NAME E(db.revision_exists(rid), origin::user, F("no such revision '%s'") % hrid); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); project.put_cert(keys, rid, typecast_vocab(idx(args, 1)), ============================================================ --- cmd_db.cc 0875bc8882c861722b83089736d55d53469fa0c7 +++ cmd_db.cc 45ddd81bad639a7a1d2f835c0c9d6a2d29b71cd0 @@ -293,7 +293,7 @@ CMD(db_changesetify, "changesetify", "", db.check_is_not_rosterified(); // early short-circuit to avoid failure after lots of work - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); build_changesets_from_manifest_ancestry(db, keys, project, set()); } @@ -314,7 +314,7 @@ CMD(db_rosterify, "rosterify", "", CMD_R db.check_is_not_rosterified(); // early short-circuit to avoid failure after lots of work - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); build_roster_style_revs_from_manifest_style_revs(db, keys, project, app.opts.attrs_to_drop); ============================================================ --- cmd_key_cert.cc a411628241629ecb7dcfbc490502701b62a63321 +++ cmd_key_cert.cc e7798f640fc449740946bbc15f3a988f7570e344 @@ -193,7 +193,7 @@ CMD(cert, "cert", "", CMD_REF(key_and_ce cert_name cname = typecast_vocab(idx(args, 1)); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); cert_value val; if (args.size() == 3) @@ -281,7 +281,7 @@ CMD(tag, "tag", "", CMD_REF(review), N_( revision_id r; complete(app.opts, app.lua, project, idx(args, 0)(), r); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); project.put_tag(keys, r, idx(args, 1)()); } @@ -302,7 +302,7 @@ CMD(testresult, "testresult", "", CMD_RE revision_id r; complete(app.opts, app.lua, project, idx(args, 0)(), r); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); project.put_revision_testresult(keys, r, idx(args, 1)()); } @@ -327,7 +327,7 @@ CMD(approve, "approve", "", CMD_REF(revi E(!app.opts.branch().empty(), origin::user, F("need --branch argument for approval")); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); project.put_revision_in_branch(keys, r, app.opts.branch); updater.maybe_do_update(); @@ -353,7 +353,7 @@ CMD(suspend, "suspend", "", CMD_REF(revi E(!app.opts.branch().empty(), origin::user, F("need --branch argument to suspend")); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); project.suspend_revision_in_branch(keys, r, app.opts.branch); updater.maybe_do_update(); @@ -390,7 +390,7 @@ CMD(comment, "comment", "", CMD_REF(revi revision_id r; complete(app.opts, app.lua, project, idx(args, 0)(), r); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); project.put_revision_comment(keys, r, comment); } ============================================================ --- cmd_merging.cc ed3530c95d77f9ec3d2f9a1fdcfabca9bc05e10c +++ cmd_merging.cc 1ee002f467d430ba43200e7f614cdf7e571049f5 @@ -527,7 +527,7 @@ CMD(merge, "merge", "", CMD_REF(tree), " % heads.size() % app.opts.branch); // avoid failure after lots of work - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); size_t pass = 1, todo = heads.size() - 1; @@ -654,7 +654,7 @@ CMD(merge_into_dir, "merge_into_dir", "" return; } - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); P(F("propagating %s -> %s") % idx(args,0) % idx(args,1)); P(F("[left] %s") % *src_i); @@ -919,7 +919,7 @@ CMD(explicit_merge, "explicit_merge", "" % right % left); // avoid failure after lots of work - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); merge_two(app.opts, app.lua, project, keys, left, right, branch, string("explicit merge"), std::cout, false); ============================================================ --- cmd_netsync.cc 689f594c47cfc766e65df0e93159ec1f121ff800 +++ cmd_netsync.cc 5f30be53c96420d71a67c75e98897c1532059b8c @@ -13,7 +13,7 @@ #include "automate_ostream_demuxed.hh" #include "merge_content.hh" -#include "netcmd.hh" +#include "netsync.hh" #include "network/connection_info.hh" #include "file_io.hh" #include "globish.hh" @@ -30,7 +30,6 @@ #include "work.hh" #include "database.hh" #include "roster.hh" -#include "vocab_cast.hh" #include "ui.hh" #include @@ -44,235 +43,63 @@ using boost::shared_ptr; using boost::shared_ptr; -static const var_key default_server_key(var_domain("database"), - var_name("default-server")); -static const var_key default_include_pattern_key(var_domain("database"), - var_name("default-include-pattern")); -static const var_key default_exclude_pattern_key(var_domain("database"), - var_name("default-exclude-pattern")); - static void -find_key(options & opts, - database & db, - key_store & keys, - lua_hooks & lua, - project_t & project, - netsync_connection_info const & info, - bool need_key = true) +extract_client_connection_info(options & opts, + project_t & project, + key_store & keys, + lua_hooks & lua, + args_vector const & args, + shared_conn_info & info, + key_requiredness_flag key_requiredness = key_required) { - utf8 host(info.client.unparsed); - if (!info.client.uri.host.empty()) - host = utf8(info.client.uri.resource, origin::user); - - cache_netsync_key(opts, db, keys, lua, project, host, - info.client.include_pattern, - info.client.exclude_pattern, - need_key ? KEY_REQUIRED : KEY_OPTIONAL); -} - -static void -build_client_connection_info(options & opts, - lua_hooks & lua, - database & db, - key_store & keys, - project_t & project, - netsync_connection_info & info, - bool address_given, - bool include_or_exclude_given, - bool need_key = true) -{ - // Use the default values if needed and available. - if (!address_given) + if (opts.remote_stdio_host_given) { - E(db.var_exists(default_server_key), origin::user, - F("no server given and no default server set")); - var_value addr_value; - db.get_var(default_server_key, addr_value); - info.client.unparsed = typecast_vocab(addr_value); - L(FL("using default server address: %s") % info.client.unparsed); + netsync_connection_info::setup_from_uri(opts, project.db, lua, + opts.remote_stdio_host, info); } - parse_uri(info.client.unparsed(), info.client.uri, origin::user); - - var_key server_include(var_domain("server-include"), - var_name(info.client.uri.resource, origin::user)); - var_key server_exclude(var_domain("server-exclude"), - var_name(info.client.uri.resource, origin::user)); - - if (info.client.uri.query.empty() && !include_or_exclude_given) + else { - if (db.var_exists(server_include)) + if (args.size() == 1) { - var_value pattern_value; - db.get_var(server_include, pattern_value); - info.client.include_pattern = globish(pattern_value(), origin::user); - L(FL("using default branch include pattern: '%s'") - % info.client.include_pattern); - if (db.var_exists(server_exclude)) - { - db.get_var(server_exclude, pattern_value); - info.client.exclude_pattern = globish(pattern_value(), origin::user); - } - else - info.client.exclude_pattern = globish(); - L(FL("excluding: %s") % info.client.exclude_pattern); + E(!opts.exclude_given, origin::user, + F("cannot use --exclude in URL mode")); + + netsync_connection_info::setup_from_uri(opts, project.db, lua, + idx(args, 0), info); } - else + else if (args.size() >= 2) { - // No include/exclude given anywhere, use the defaults. - E(db.var_exists(default_include_pattern_key), origin::user, - F("no branch pattern given and no default pattern set")); - var_value pattern_value; - db.get_var(default_include_pattern_key, pattern_value); - info.client.include_pattern = globish(pattern_value(), origin::user); - L(FL("using default branch include pattern: '%s'") - % info.client.include_pattern); - if (db.var_exists(default_exclude_pattern_key)) - { - db.get_var(default_exclude_pattern_key, pattern_value); - info.client.exclude_pattern = globish(pattern_value(), origin::user); - } - else - info.client.exclude_pattern = globish(); - L(FL("excluding: %s") % info.client.exclude_pattern); - } - } - else if(!info.client.uri.query.empty()) - { - E(!include_or_exclude_given, origin::user, - F("Include/exclude pattern was given both as part of the URL and as a separate argument.")); + arg_type server = idx(args, 0); + vector include_patterns; + include_patterns.insert(include_patterns.begin(), + args.begin() + 1, + args.end()); + vector exclude_patterns = opts.exclude_patterns; - // Pull include/exclude from the query string - char const separator = ','; - char const negate = '-'; - string const & query(info.client.uri.query); - std::vector includes, excludes; - string::size_type begin = 0; - string::size_type end = query.find(separator); - while (begin < query.size()) - { - std::string item = query.substr(begin, end); - if (end == string::npos) - begin = end; - else - { - begin = end+1; - if (begin < query.size()) - end = query.find(separator, begin); - } - - bool is_exclude = false; - if (item.size() >= 1 && item.at(0) == negate) - { - is_exclude = true; - item.erase(0, 1); - } - else if (item.find("include=") == 0) - { - item.erase(0, string("include=").size()); - } - else if (item.find("exclude=") == 0) - { - is_exclude = true; - item.erase(0, string("exclude=").size()); - } - - if (is_exclude) - excludes.push_back(arg_type(urldecode(item, origin::user), - origin::user)); - else - includes.push_back(arg_type(urldecode(item, origin::user), - origin::user)); + netsync_connection_info::setup_from_server_and_pattern(opts, project.db, + lua, server, + include_patterns, + exclude_patterns, + info); } - info.client.include_pattern = globish(includes); - info.client.exclude_pattern = globish(excludes); + else + { + // if no argument has been given and the --remote_stdio_host + // option has been left out, try to load the database defaults + // at least + netsync_connection_info::setup_default(opts, project.db, lua, info); + } } - // Maybe set the default values. - if (!db.var_exists(default_server_key) - || opts.set_default) - { - P(F("setting default server to %s") % info.client.uri.resource); - db.set_var(default_server_key, - var_value(info.client.uri.resource, origin::user)); - } - if (!db.var_exists(default_include_pattern_key) - || opts.set_default) - { - P(F("setting default branch include pattern to '%s'") - % info.client.include_pattern); - db.set_var(default_include_pattern_key, - typecast_vocab(info.client.include_pattern)); - } - if (!db.var_exists(default_exclude_pattern_key) - || opts.set_default) - { - P(F("setting default branch exclude pattern to '%s'") - % info.client.exclude_pattern); - db.set_var(default_exclude_pattern_key, - typecast_vocab(info.client.exclude_pattern)); - } - if (!db.var_exists(server_include) - || opts.set_default) - { - P(F("setting default include pattern for server '%s' to '%s'") - % info.client.uri.resource % info.client.include_pattern); - db.set_var(server_include, - typecast_vocab(info.client.include_pattern)); - } - if (!db.var_exists(server_exclude) - || opts.set_default) - { - P(F("setting default exclude pattern for server '%s' to '%s'") - % info.client.uri.resource % info.client.exclude_pattern); - db.set_var(server_exclude, - typecast_vocab(info.client.exclude_pattern)); - } + opts.use_transport_auth = + lua.hook_use_transport_auth(info->client.get_uri()); - info.client.use_argv = - lua.hook_get_netsync_connect_command(info.client.uri, - info.client.include_pattern, - info.client.exclude_pattern, - global_sanity.debug_p(), - info.client.argv); - opts.use_transport_auth = lua.hook_use_transport_auth(info.client.uri); if (opts.use_transport_auth) { - find_key(opts, db, keys, lua, project, info, need_key); + cache_netsync_key(opts, project, keys, lua, info, key_requiredness); } - - info.client.connection_type = netsync_connection_info::netsync_connection; } -static void -extract_client_connection_info(options & opts, - lua_hooks & lua, - database & db, - key_store & keys, - project_t & project, - args_vector const & args, - netsync_connection_info & info, - bool need_key = true) -{ - bool have_address = false; - bool have_include_exclude = false; - if (args.size() >= 1) - { - have_address = true; - info.client.unparsed = idx(args, 0); - } - if (args.size() >= 2 || opts.exclude_given) - { - E(args.size() >= 2, origin::user, F("no branch pattern given")); - - have_include_exclude = true; - info.client.include_pattern = globish(args.begin() + 1, args.end()); - info.client.exclude_pattern = globish(opts.exclude_patterns); - } - build_client_connection_info(opts, lua, db, keys, project, - info, have_address, have_include_exclude, - need_key); -} - CMD_AUTOMATE_NO_STDIO(remote_stdio, N_("[ADDRESS[:PORTNUMBER]]"), N_("Opens an 'automate stdio' connection to a remote server"), @@ -298,48 +125,13 @@ CMD_AUTOMATE_NO_STDIO(remote_stdio, key_store keys(app); project_t project(db); - netsync_connection_info info; + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, info); - if (args.size() == 1) - { - info.client.unparsed = idx(args, 0); - } - else - { - E(db.var_exists(default_server_key), origin::user, - F("no server given and no default server set")); - var_value addr_value; - db.get_var(default_server_key, addr_value); - info.client.unparsed = typecast_vocab(addr_value); - L(FL("using default server address: %s") % info.client.unparsed); - } - - parse_uri(info.client.unparsed(), info.client.uri, origin::user); - - if (!db.var_exists(default_server_key) || app.opts.set_default) - { - P(F("setting default server to %s") % info.client.uri.resource); - db.set_var(default_server_key, - var_value(info.client.uri.resource, origin::user)); - } - - info.client.use_argv = - app.lua.hook_get_netsync_connect_command(info.client.uri, - info.client.include_pattern, - info.client.exclude_pattern, - global_sanity.debug_p(), - info.client.argv); - app.opts.use_transport_auth = app.lua.hook_use_transport_auth(info.client.uri); - if (app.opts.use_transport_auth) - { - find_key(app.opts, db, keys, app.lua, project, info, true); - } - - info.client.connection_type = netsync_connection_info::automate_connection; - - info.client.set_input_stream(std::cin); + info->client.set_connection_type(netsync_connection_info::automate_connection); + info->client.set_input_stream(std::cin); automate_ostream os(output, app.opts.automate_stdio_size); - info.client.set_output_stream(os); + info->client.set_output_stream(os); run_netsync_protocol(app, app.opts, app.lua, project, keys, client_voice, source_and_sink_role, info); @@ -430,43 +222,10 @@ CMD_AUTOMATE_NO_STDIO(remote, key_store keys(app); project_t project(db); - netsync_connection_info info; + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, + args_vector(), info); - if (app.opts.remote_stdio_host_given) - { - info.client.unparsed = app.opts.remote_stdio_host; - } - else - { - E(db.var_exists(default_server_key), origin::user, - F("no server given and no default server set")); - var_value addr_value; - db.get_var(default_server_key, addr_value); - info.client.unparsed = typecast_vocab(addr_value); - L(FL("using default server address: %s") % info.client.unparsed); - } - - parse_uri(info.client.unparsed(), info.client.uri, origin::user); - - if (!db.var_exists(default_server_key) || app.opts.set_default) - { - P(F("setting default server to %s") % info.client.uri.resource); - db.set_var(default_server_key, - var_value(info.client.uri.resource, origin::user)); - } - - info.client.use_argv = - app.lua.hook_get_netsync_connect_command(info.client.uri, - info.client.include_pattern, - info.client.exclude_pattern, - global_sanity.debug_p(), - info.client.argv); - app.opts.use_transport_auth = app.lua.hook_use_transport_auth(info.client.uri); - if (app.opts.use_transport_auth) - { - find_key(app.opts, db, keys, app.lua, project, info, true); - } - args_vector cleaned_args(args); std::vector > opts; parse_options_from_args(cleaned_args, opts); @@ -493,11 +252,11 @@ CMD_AUTOMATE_NO_STDIO(remote, L(FL("stdio input: %s") % ss.str()); - info.client.set_input_stream(ss); automate_ostream_demuxed os(output, std::cerr, app.opts.automate_stdio_size); - info.client.set_output_stream(os); - info.client.connection_type = netsync_connection_info::automate_connection; + info->client.set_connection_type(netsync_connection_info::automate_connection); + info->client.set_input_stream(ss); + info->client.set_output_stream(os); run_netsync_protocol(app, app.opts, app.lua, project, keys, client_voice, source_and_sink_role, info); @@ -519,9 +278,8 @@ CMD(push, "push", "", CMD_REF(network), key_store keys(app); project_t project(db); - netsync_connection_info info; - extract_client_connection_info(app.opts, app.lua, db, keys, - project, args, info); + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, info); run_netsync_protocol(app, app.opts, app.lua, project, keys, client_voice, source_role, info); @@ -539,8 +297,8 @@ CMD_AUTOMATE(push, N_("[ADDRESS[:PORTNUM key_store keys(app); project_t project(db); - netsync_connection_info info; - extract_client_connection_info(app.opts, app.lua, db, keys, project, args, info); + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, info); run_netsync_protocol(app, app.opts, app.lua, project, keys, client_voice, source_role, info); @@ -561,9 +319,9 @@ CMD(pull, "pull", "", CMD_REF(network), maybe_workspace_updater updater(app, project); - netsync_connection_info info; - extract_client_connection_info(app.opts, app.lua, db, keys, project, - args, info, false); + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, info, + key_optional); if (!keys.have_signing_key()) P(F("doing anonymous pull; use -kKEYNAME if you need authentication")); @@ -585,9 +343,9 @@ CMD_AUTOMATE(pull, N_("[ADDRESS[:PORTNUM key_store keys(app); project_t project(db); - netsync_connection_info info; - extract_client_connection_info(app.opts, app.lua, db, keys, project, - args, info, false); + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, + info, key_optional); run_netsync_protocol(app, app.opts, app.lua, project, keys, client_voice, sink_role, info); @@ -608,9 +366,8 @@ CMD(sync, "sync", "", CMD_REF(network), maybe_workspace_updater updater(app, project); - netsync_connection_info info; - extract_client_connection_info(app.opts, app.lua, db, keys, - project, args, info); + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, info); if (app.opts.set_default && workspace::found) { @@ -636,8 +393,8 @@ CMD_AUTOMATE(sync, N_("[ADDRESS[:PORTNUM key_store keys(app); project_t project(db); - netsync_connection_info info; - extract_client_connection_info(app.opts, app.lua, db, keys, project, args, info); + shared_conn_info info; + extract_client_connection_info(app.opts, project, keys, app.lua, args, info); if (app.opts.set_default && workspace::found) { @@ -651,7 +408,7 @@ CMD(clone, "clone", "", CMD_REF(network) } CMD(clone, "clone", "", CMD_REF(network), - N_("ADDRESS[:PORTNUMBER] BRANCH [DIRECTORY]"), + N_("HOST[:PORTNUMBER] BRANCH [DIRECTORY], URL [DIRECTORY]"), N_("Checks out a revision from a remote database into a directory"), N_("If a revision is given, that's the one that will be checked out. " "Otherwise, it will be the head of the branch supplied. " @@ -659,31 +416,85 @@ CMD(clone, "clone", "", CMD_REF(network) options::opts::max_netsync_version | options::opts::min_netsync_version | options::opts::revision) { - if (args.size() < 2 || args.size() > 3 || app.opts.revision_selectors.size() > 1) + + bool url_arg = (args.size() == 1 || args.size() == 2) && + idx(args, 0)().find("://") != string::npos; + + bool host_branch_arg = (args.size() == 2 || args.size() == 3) && + idx(args, 0)().find("://") == string::npos; + + bool no_ambigious_revision = app.opts.revision_selectors.size() < 2; + + if (!(no_ambigious_revision && (url_arg || host_branch_arg))) throw usage(execid); - revision_id ident; - system_path workspace_dir; - netsync_connection_info info; - info.client.unparsed = idx(args, 0); + // we create the database before anything else, but we + // do not clean newly created databases up if the clone fails + // (and I think this is correct, because if the pull fails later + // on due to some network error, the use does not have to start + // again from the beginning) + database_path_helper helper(app.lua); + helper.maybe_set_default_alias(app.opts); - branch_name branchname = typecast_vocab(idx(args, 1)); + database db(app); + project_t project(db); + key_store keys(app); - E(!branchname().empty(), origin::user, - F("you must specify a branch to clone")); + db.create_if_not_exists(); + db.ensure_open(); + shared_conn_info info; + arg_type server = idx(args, 0); + arg_type workspace_arg; + + if (url_arg) + { + E(!app.opts.exclude_given, origin::user, + F("cannot use --exclude in URL mode")); + + netsync_connection_info::setup_from_uri(app.opts, project.db, app.lua, + server, info); + if (args.size() == 2) + workspace_arg = idx(args, 1); + } + else + { + vector include_patterns; + include_patterns.push_back(idx(args, 1)); + netsync_connection_info::setup_from_server_and_pattern(app.opts, project.db, + app.lua, server, + include_patterns, + app.opts.exclude_patterns, + info); + if (args.size() == 3) + workspace_arg = idx(args, 2); + } + + globish include_pattern = info->client.get_include_pattern(); + E(!include_pattern().empty() && !include_pattern.contains_meta_chars(), + origin::user, F("you must specify an unambiguous branch to clone")); + branch_name branchname(include_pattern(), origin::user); + + app.opts.use_transport_auth = + app.lua.hook_use_transport_auth(info->client.get_uri()); + + if (app.opts.use_transport_auth) + { + cache_netsync_key(app.opts, project, keys, app.lua, info, key_optional); + } + bool target_is_current_dir = false; - if (args.size() == 2) + system_path workspace_dir; + if (workspace_arg().empty()) { // No checkout dir specified, use branch name for dir. workspace_dir = system_path(branchname(), origin::user); } else { - // Checkout to specified dir. - workspace_dir = system_path(idx(args, 2)); - if (idx(args, 2) == utf8(".")) - target_is_current_dir = true; + target_is_current_dir = + workspace_arg == utf8("."); + workspace_dir = system_path(workspace_arg); } if (!target_is_current_dir) @@ -708,13 +519,6 @@ CMD(clone, "clone", "", CMD_REF(network) // db URIs will work system_path start_dir(get_current_working_dir(), origin::system); - database_path_helper helper(app.lua); - helper.maybe_set_default_alias(app.opts); - - database db(app); - db.create_if_not_exists(); - db.ensure_open(); - // this is actually stupid, but app.opts.branch must be set here // otherwise it will not be written into _MTN/options, in case // a revision is chosen which has multiple branch certs @@ -722,15 +526,6 @@ CMD(clone, "clone", "", CMD_REF(network) workspace::create_workspace(app.opts, app.lua, workspace_dir); app.opts.branch = branch_name(); - key_store keys(app); - project_t project(db); - - info.client.include_pattern = globish(branchname(), origin::user); - info.client.exclude_pattern = globish(app.opts.exclude_patterns); - - build_client_connection_info(app.opts, app.lua, db, keys, project, - info, true, true, false); - if (!keys.have_signing_key()) P(F("doing anonymous pull; use -kKEYNAME if you need authentication")); @@ -744,6 +539,7 @@ CMD(clone, "clone", "", CMD_REF(network) transaction_guard guard(db, false); + revision_id ident; if (app.opts.revision_selectors.empty()) { set heads; @@ -844,24 +640,13 @@ CMD_NO_WORKSPACE(serve, "serve", "", CMD db.ensure_open(); - netsync_connection_info info; - info.server.addrs = app.opts.bind_uris; + shared_conn_info info; + netsync_connection_info::setup_for_serve(app.opts, project.db, app.lua, info); if (app.opts.use_transport_auth) { - E(app.lua.hook_persist_phrase_ok(), origin::user, - F("need permission to store persistent passphrase " - "(see hook persist_phrase_ok())")); - - info.client.include_pattern = globish("*", origin::internal); - info.client.exclude_pattern = globish("", origin::internal); - if (!app.opts.bind_uris.empty()) - info.client.unparsed = *app.opts.bind_uris.begin(); - find_key(app.opts, db, keys, app.lua, project, info); + cache_netsync_key(app.opts, project, keys, app.lua, info, key_required); } - else if (!app.opts.bind_stdio) - W(F("The --no-transport-auth option is usually only used " - "in combination with --stdio")); run_netsync_protocol(app, app.opts, app.lua, project, keys, server_voice, source_and_sink_role, info); ============================================================ --- cmd_othervcs.cc 6894f2644573582dadfeccb713ae3362c2953189 +++ cmd_othervcs.cc ba717ebc34573dc41258fe4f679632876097eb48 @@ -63,7 +63,7 @@ CMD(cvs_import, "cvs_import", "", CMD_RE // make sure we can sign certs using the selected key; also requests // the password (if necessary) up front rather than after some arbitrary // amount of work - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); import_cvs_repo(project, keys, cvsroot, app.opts.branch); } @@ -73,7 +73,7 @@ CMD(git_export, "git_export", "", CMD_RE N_("Produces a git fast-export data stream on stdout"), (""), options::opts::authors_file | options::opts::branches_file | - options::opts::log_revids | options::opts::log_certs | + options::opts::log_revids | options::opts::log_certs | options::opts::use_one_changelog | options::opts::import_marks | options::opts::export_marks | options::opts::refs) ============================================================ --- cmd_ws_commit.cc 729a6ead62f91c4f1c8629938f15b5cfba69c5f4 +++ cmd_ws_commit.cc fc8e63fddea457889a603521c09cd199fb997bef @@ -610,7 +610,7 @@ CMD(disapprove, "disapprove", "", CMD_RE % r).str(), origin::internal)); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); // for the divergence check, below set heads; @@ -1541,7 +1541,7 @@ CMD(commit, "commit", "ci", CMD_REF(work E(message_validated, origin::user, F("log message rejected by hook: %s") % reason); - cache_user_key(app.opts, app.lua, db, keys, project); + cache_user_key(app.opts, project, keys, app.lua); // for the divergence check, below set heads; ============================================================ --- globish.cc 338d0214a12f6adaf4bda966e495fefb91e3a054 +++ globish.cc 1a7b2b42320d0c8d8fde56b5af871bc125d17a5a @@ -326,6 +326,26 @@ globish::operator()() const return decode(compiled_pattern.begin(), compiled_pattern.end()); } +bool +globish::contains_meta_chars() const +{ + string::const_iterator p = compiled_pattern.begin(); + for (; p != compiled_pattern.end(); p++) + switch (*p) + { + case META_STAR: + case META_QUES: + case META_CC_BRA: + case META_CC_KET: + case META_CC_INV_BRA: + case META_ALT_BRA: + case META_ALT_KET: + case META_ALT_OR: + return true; + } + return false; +} + template <> void dump(globish const & g, string & s) { s = g(); ============================================================ --- globish.hh 864db1e5341486754cbb90b73eca6e4e0886f1d6 +++ globish.hh 93fca3ce5259a69e5ad8020c6d51560a696de54b @@ -59,6 +59,7 @@ struct globish : origin_aware std::vector::const_iterator const & end); std::string operator()(void) const; + bool contains_meta_chars() const; bool matches(std::string const & target) const; private: ============================================================ --- keys.cc e27fd43f8730dc5ada1f4aaddfb55e910ac659ac +++ keys.cc 49e561d782572c02d86f9592e640d67b46cb9948 @@ -22,6 +22,9 @@ #include "project.hh" #include "key_store.hh" #include "database.hh" +#include "uri.hh" +#include "globish.hh" +#include "network/connection_info.hh" using std::string; using std::vector; @@ -91,14 +94,14 @@ namespace { // Decrypt and cache the key now. keys.cache_decrypted_key(chosen_key); } - bool get_only_key(key_store & keys, bool required, key_id & key) + bool get_only_key(key_store & keys, key_requiredness_flag key_requiredness, key_id & key) { vector all_privkeys; keys.get_key_ids(all_privkeys); - E(!required || !all_privkeys.empty(), origin::user, + E(key_requiredness == key_optional || !all_privkeys.empty(), origin::user, F("you have no private key to make signatures with\n" "perhaps you need to 'genkey '")); - E(!required || all_privkeys.size() < 2, origin::user, + E(key_requiredness == key_optional || all_privkeys.size() < 2, origin::user, F("you have multiple private keys\n" "pick one to use for signatures by adding " "'-k' to your command")); @@ -147,7 +150,7 @@ get_user_key(options const & opts, lua_h ; // the lua hook sets the key else { - get_only_key(keys, true, key); + get_only_key(keys, key_required, key); } if (cache == cache_enable) @@ -156,14 +159,11 @@ cache_netsync_key(options const & opts, void cache_netsync_key(options const & opts, - database & db, + project_t & project, key_store & keys, lua_hooks & lua, - project_t & project, - utf8 const & host, - globish const & include, - globish const & exclude, - netsync_key_requiredness key_requiredness) + shared_conn_info const & info, + key_requiredness_flag key_requiredness) { if (keys.have_signing_key()) { @@ -185,28 +185,32 @@ cache_netsync_key(options const & opts, found_key = true; } } - else if (lua.hook_get_netsync_key(host, include, exclude, keys, project, key)) + else if (lua.hook_get_netsync_key(utf8(info->client.get_uri().resource, origin::user), + info->client.get_include_pattern(), + info->client.get_exclude_pattern(), + keys, project, key)) { found_key = true; } else { - found_key = get_only_key(keys, key_requiredness == KEY_REQUIRED, key); + found_key = get_only_key(keys, key_requiredness, key); } if (found_key) { - check_and_save_chosen_key(db, keys, key); + check_and_save_chosen_key(project.db, keys, key); } } void -cache_user_key(options const & opts, lua_hooks & lua, - database & db, key_store & keys, - project_t & project) +cache_user_key(options const & opts, + project_t & project, + key_store & keys, + lua_hooks & lua) { key_id key; - get_user_key(opts, lua, db, keys, project, key); + get_user_key(opts, lua, project.db, keys, project, key); } void ============================================================ --- keys.hh 6b5412008a7158b8abfb8228163b687538d7879f +++ keys.hh 8b5f0676693fa00e68f7132e50abbc22b5097252 @@ -20,7 +20,11 @@ class globish; struct keypair; class globish; +class netsync_connection_info; +typedef boost::shared_ptr shared_conn_info; + enum key_cache_flag { cache_disable, cache_enable }; +enum key_requiredness_flag {key_optional, key_required}; // keys.{hh,cc} does all the "delicate" crypto (meaning: that which needs // to read passphrases and manipulate raw, decrypted private keys). it @@ -30,48 +34,52 @@ enum key_cache_flag { cache_disable, cac // Find the key to be used for signing certs. If possible, ensure the // database and the key_store agree on that key, and optionally cache it in // decrypted form, so as not to bother the user for their passphrase later. -void get_user_key(options const & opts, lua_hooks & lua, - database & db, key_store & keys, - project_t & project, key_id & key, - key_cache_flag const cache = cache_enable); +void +get_user_key(options const & opts, lua_hooks & lua, + database & db, key_store & keys, + project_t & project, key_id & key, + key_cache_flag const cache = cache_enable); // As above, but does not report which key has been selected; for use when // the important thing is to have selected one and cached the decrypted key. -void cache_user_key(options const & opts, lua_hooks & lua, - database & db, key_store & keys, - project_t & project); +void +cache_user_key(options const & opts, + project_t & project, + key_store & keys, + lua_hooks & lua); // Find the key to be used for netsync authentication. If possible, ensure the // database and the key_store agree on that key, and cache it in decrypted // form, so as not to bother the user for their passphrase later. -enum netsync_key_requiredness {KEY_OPTIONAL, KEY_REQUIRED}; -void cache_netsync_key(options const & opts, - database & db, - key_store & keys, - lua_hooks & lua, - project_t & project, - utf8 const & host, - globish const & include, - globish const & exclude, - netsync_key_requiredness key_requiredness); +void +cache_netsync_key(options const & opts, + project_t & project, + key_store & keys, + lua_hooks & lua, + shared_conn_info const & info, + key_requiredness_flag key_requiredness); -void load_key_pair(key_store & keys, - key_id const & id); +void +load_key_pair(key_store & keys, + key_id const & id); -void load_key_pair(key_store & keys, - key_id const & id, - keypair & kp); +void +load_key_pair(key_store & keys, + key_id const & id, + keypair & kp); // netsync stuff -void key_hash_code(key_name const & ident, - rsa_pub_key const & pub, - key_id & out); +void +key_hash_code(key_name const & ident, + rsa_pub_key const & pub, + key_id & out); -bool keys_match(key_name const & id1, - rsa_pub_key const & key1, - key_name const & id2, - rsa_pub_key const & key2); +bool +keys_match(key_name const & id1, + rsa_pub_key const & key1, + key_name const & id2, + rsa_pub_key const & key2); #endif // __KEYS_HH__ ============================================================ --- netcmd.hh f6c60950810184737024220f58f806db5ec1ee2a +++ netcmd.hh 6ac2e5bcf92deae29cbad403b4a0397f0a882734 @@ -232,15 +232,6 @@ public: }; -class netsync_connection_info; - -void run_netsync_protocol(app_state & app, - options & opts, lua_hooks & lua, - project_t & project, key_store & keys, - protocol_voice voice, - protocol_role role, - netsync_connection_info const & info); - #endif // __NETCMD_HH__ // Local Variables: ============================================================ --- netsync.cc dd63fc06d6bab3dd3e8c30a1df6c23975bf94a54 +++ netsync.cc 7d2cf5774d36b4fca9c1434f54f60a424f39f7db @@ -9,6 +9,7 @@ // PURPOSE. #include "base.hh" +#include "netsync.hh" #include @@ -34,13 +35,6 @@ using boost::shared_ptr; using boost::lexical_cast; using boost::shared_ptr; -struct server_initiated_sync_request -{ - string what; - string address; - string include; - string exclude; -}; deque server_initiated_sync_requests; LUAEXT(server_request_sync, ) { @@ -48,11 +42,20 @@ LUAEXT(server_request_sync, ) char const * a = luaL_checkstring(LS, 2); char const * i = luaL_checkstring(LS, 3); char const * e = luaL_checkstring(LS, 4); + server_initiated_sync_request request; - request.what = string(w); request.address = string(a); request.include = string(i); request.exclude = string(e); + + request.role = source_and_sink_role; + if (w == "sync") + request.role = source_and_sink_role; + else if (w == "push") + request.role = source_role; + else if (w == "pull") + request.role = sink_role; + server_initiated_sync_requests.push_back(request); return 0; } @@ -60,19 +63,19 @@ build_stream_to_server(options & opts, l static shared_ptr build_stream_to_server(options & opts, lua_hooks & lua, - netsync_connection_info info, - Netxx::port_type default_port, + shared_conn_info const & info, Netxx::Timeout timeout) { shared_ptr server; - if (info.client.use_argv) + if (info->client.get_use_argv()) { - I(!info.client.argv.empty()); - string cmd = info.client.argv[0]; - info.client.argv.erase(info.client.argv.begin()); + vector args = info->client.get_argv(); + I(!args.empty()); + string cmd = args[0]; + args.erase(args.begin()); return shared_ptr - (new Netxx::PipeStream(cmd, info.client.argv)); + (new Netxx::PipeStream(cmd, args)); } else { @@ -81,12 +84,11 @@ build_stream_to_server(options & opts, l #else bool use_ipv6=false; #endif - string host(info.client.uri.host); + string host(info->client.get_uri().host); I(!host.empty()); - if (!info.client.uri.port.empty()) - default_port = lexical_cast(info.client.uri.port); Netxx::Address addr(host.c_str(), - default_port, use_ipv6); + info->client.get_port(), + use_ipv6); return shared_ptr (new Netxx::Stream(addr, timeout)); } @@ -97,7 +99,7 @@ call_server(app_state & app, project_t & project, key_store & keys, protocol_role role, - netsync_connection_info const & info) + shared_conn_info const & info) { Netxx::PipeCompatibleProbe probe; transaction_guard guard(project.db); @@ -105,14 +107,11 @@ call_server(app_state & app, Netxx::Timeout timeout(static_cast(constants::netsync_timeout_seconds)), instant(0,1); - P(F("connecting to %s") % info.client.uri.host); + P(F("connecting to %s") % info->client.get_uri().resource); shared_ptr server - = build_stream_to_server(app.opts, app.lua, - info, info.client.get_port(), - timeout); + = build_stream_to_server(app.opts, app.lua, info, timeout); - // 'false' here means not to revert changes when the SockOpt // goes out of scope. Netxx::SockOpt socket_options(server->get_socketfd(), false); @@ -120,21 +119,21 @@ call_server(app_state & app, shared_ptr sess(new session(app, project, keys, client_voice, - info.client.uri.resource, server)); + info->client.get_uri().resource, server)); shared_ptr wrapped; - switch (info.client.connection_type) + switch (info->client.get_connection_type()) { case netsync_connection_info::netsync_connection: wrapped.reset(new netsync_session(sess.get(), app.opts, app.lua, project, keys, role, - info.client.include_pattern, - info.client.exclude_pattern)); + info->client.get_include_pattern(), + info->client.get_exclude_pattern())); break; case netsync_connection_info::automate_connection: wrapped.reset(new automate_session(app, sess.get(), - &info.client.get_input_stream(), - &info.client.get_output_stream())); + &info->client.get_input_stream(), + &info->client.get_output_stream())); break; } sess->set_inner(wrapped); @@ -205,20 +204,16 @@ session_from_server_sync_item(app_state key_store & keys, server_initiated_sync_request const & request) { - netsync_connection_info info; - info.client.unparsed = utf8(request.address, origin::user); - info.client.include_pattern = globish(request.include, origin::user); - info.client.exclude_pattern = globish(request.exclude, origin::user); - info.client.use_argv = false; - parse_uri(info.client.unparsed(), info.client.uri, - origin::user /* from lua hook */); + shared_conn_info info; + netsync_connection_info::setup_from_sync_request(app.opts, project.db, + app.lua, request, + info); try { - P(F("connecting to %s") % info.client.uri.host); + P(F("connecting to %s") % info->client.get_uri().resource); shared_ptr server - = build_stream_to_server(app.opts, app.lua, - info, info.client.get_port(), + = build_stream_to_server(app.opts, app.lua, info, Netxx::Timeout(constants::netsync_timeout_seconds)); // 'false' here means not to revert changes when @@ -226,24 +221,16 @@ session_from_server_sync_item(app_state Netxx::SockOpt socket_options(server->get_socketfd(), false); socket_options.set_non_blocking(); - protocol_role role = source_and_sink_role; - if (request.what == "sync") - role = source_and_sink_role; - else if (request.what == "push") - role = source_role; - else if (request.what == "pull") - role = sink_role; - shared_ptr sess(new session(app, project, keys, client_voice, - info.client.uri.resource, server)); + info->client.get_uri().resource, server)); shared_ptr wrapped(new netsync_session(sess.get(), app.opts, app.lua, project, - keys, role, - info.client.include_pattern, - info.client.exclude_pattern, + keys, request.role, + info->client.get_include_pattern(), + info->client.get_exclude_pattern(), true)); sess->set_inner(wrapped); return sess; @@ -389,20 +376,8 @@ run_netsync_protocol(app_state & app, project_t & project, key_store & keys, protocol_voice voice, protocol_role role, - netsync_connection_info const & info) + shared_conn_info & info) { - if (info.client.include_pattern().find_first_of("'\"") != string::npos) - { - W(F("include branch pattern contains a quote character:\n" - "%s") % info.client.include_pattern()); - } - - if (info.client.exclude_pattern().find_first_of("'\"") != string::npos) - { - W(F("exclude branch pattern contains a quote character:\n" - "%s") % info.client.exclude_pattern()); - } - // We do not want to be killed by SIGPIPE from a network disconnect. ignore_sigpipe(); @@ -422,13 +397,13 @@ run_netsync_protocol(app_state & app, } else serve_connections(app, opts, lua, project, keys, - role, info.server.addrs); + role, info->server.addrs); } else { I(voice == client_voice); - call_server(app, project, keys, - role, info); + call_server(app, project, keys, role, info); + info->client.set_connection_successful(); } } catch (Netxx::NetworkException & e) ============================================================ --- network/connection_info.cc f1426c49f3d6d2a8492c444f2555f5de5de75a1d +++ network/connection_info.cc ab2085f8d0836e1234588f605ff37679f3ae45bd @@ -10,44 +10,480 @@ #include "base.hh" #include "constants.hh" +#include "vocab.hh" + +#include "database.hh" +#include "options.hh" +#include "globish.hh" +#include "lexical_cast.hh" +#include "lua_hooks.hh" +#include "netsync.hh" #include "network/connection_info.hh" +#include "uri.hh" +#include "vocab_cast.hh" -netsync_connection_info::Client::Client() : +#include + +using std::vector; +using std::string; +using boost::lexical_cast; + +netsync_connection_info::netsync_connection_info(database & d, options const & o) : + client(d, o) +{ } + +netsync_connection_info::Client::Client(database & d, options const & o) : + connection_successful(false), use_argv(false), connection_type(netsync_connection), input_stream(0), - output_stream(0) -{ } + output_stream(0), + db(d), opts(o) +{ + var_key default_server_key(var_domain("database"), + var_name("default-server")); + var_key default_include_pattern_key(var_domain("database"), + var_name("default-include-pattern")); + var_key default_exclude_pattern_key(var_domain("database"), + var_name("default-exclude-pattern")); -std::istream & netsync_connection_info::Client::get_input_stream() const + if (db.var_exists(default_server_key)) + { + var_value addr_value; + db.get_var(default_server_key, addr_value); + try + { + set_raw_uri(addr_value()); + L(FL("loaded default server address: %s") % addr_value()); + } + catch (recoverable_failure & e) + { + W(F("ignoring invalid default server address '%s': %s") + % addr_value() % e.what()); + } + } + + if (db.var_exists(default_include_pattern_key)) + { + var_value pattern_value; + db.get_var(default_include_pattern_key, pattern_value); + try + { + include_pattern = globish(pattern_value(), origin::user); + L(FL("loaded default branch include pattern: '%s'") % include_pattern); + } + catch (recoverable_failure & e) + { + W(F("ignoring invalid default branch include pattern '%s': %s") + % include_pattern % e.what()); + } + + if (db.var_exists(default_exclude_pattern_key)) + { + db.get_var(default_exclude_pattern_key, pattern_value); + try + { + exclude_pattern = globish(pattern_value(), origin::user); + L(FL("loaded default branch exclude pattern: '%s'") % exclude_pattern); + } + catch (recoverable_failure & e) + { + W(F("ignoring invalid default branch exclude pattern '%s': %s") + % exclude_pattern % e.what()); + } + } + } +} + +netsync_connection_info::Client::~Client() { + if (connection_successful) + { + var_key default_server_key(var_domain("database"), + var_name("default-server")); + var_key default_include_pattern_key(var_domain("database"), + var_name("default-include-pattern")); + var_key default_exclude_pattern_key(var_domain("database"), + var_name("default-exclude-pattern")); + + var_key server_include(var_domain("server-include"), + var_name(uri.resource, origin::user)); + var_key server_exclude(var_domain("server-exclude"), + var_name(uri.resource, origin::user)); + + // Maybe set the default values. + if (!db.var_exists(default_server_key) + || opts.set_default) + { + P(F("setting default server to %s") % uri.resource); + db.set_var(default_server_key, + var_value(uri.resource, origin::user)); + } + if (!db.var_exists(default_include_pattern_key) + || opts.set_default) + { + P(F("setting default branch include pattern to '%s'") % include_pattern); + db.set_var(default_include_pattern_key, + typecast_vocab(include_pattern)); + } + if (!db.var_exists(default_exclude_pattern_key) + || opts.set_default) + { + P(F("setting default branch exclude pattern to '%s'") % exclude_pattern); + db.set_var(default_exclude_pattern_key, + typecast_vocab(exclude_pattern)); + } + if (!db.var_exists(server_include) + || opts.set_default) + { + P(F("setting default include pattern for server '%s' to '%s'") + % uri.resource % include_pattern); + db.set_var(server_include, + typecast_vocab(include_pattern)); + } + if (!db.var_exists(server_exclude) + || opts.set_default) + { + P(F("setting default exclude pattern for server '%s' to '%s'") + % uri.resource % exclude_pattern); + db.set_var(server_exclude, + typecast_vocab(exclude_pattern)); + } + } +} + +void +netsync_connection_info::Client::set_connection_successful() +{ + connection_successful = true; +} + +std::istream & +netsync_connection_info::Client::get_input_stream() const +{ I(input_stream); return *input_stream; } -automate_ostream & netsync_connection_info::Client::get_output_stream() const +automate_ostream & +netsync_connection_info::Client::get_output_stream() const { I(output_stream); return *output_stream; } -void netsync_connection_info::Client::set_input_stream(std::istream & is) +void +netsync_connection_info::Client::set_input_stream(std::istream & is) { input_stream = &is; } -void netsync_connection_info::Client::set_output_stream(automate_ostream & os) +void +netsync_connection_info::Client::set_output_stream(automate_ostream & os) { output_stream = &os; } -std::size_t netsync_connection_info::Client::get_port() const +Netxx::port_type +netsync_connection_info::Client::get_port() const { - if (uri.port.empty()) - return constants::netsync_default_port; - return atoi(uri.port.c_str()); + std::size_t port = constants::netsync_default_port; + if (!uri.port.empty()) + port = atoi(uri.port.c_str()); + return lexical_cast(port); } +void +netsync_connection_info::Client::set_include_pattern(vector const & pat) +{ + // do not overwrite default patterns + if (pat.size() == 0) + return; + + for (vector::const_iterator p = pat.begin(); p != pat.end(); p++) + { + if ((*p)().find_first_of("'\"") != string::npos) + { + W(F("include branch pattern contains a quote character:\n" + "%s") % (*p)()); + } + } + include_pattern = globish(pat); +} + +globish +netsync_connection_info::Client::get_include_pattern() const +{ + return include_pattern; +} + +void +netsync_connection_info::Client::set_exclude_pattern(vector const & pat) +{ + // do not overwrite default patterns + if (pat.size() == 0) + return; + + for (vector::const_iterator p = pat.begin(); p != pat.end(); p++) + { + if ((*p)().find_first_of("'\"") != string::npos) + { + W(F("exclude branch pattern contains a quote character:\n" + "%s") % (*p)()); + } + } + exclude_pattern = globish(pat); +} + +globish +netsync_connection_info::Client::get_exclude_pattern() const +{ + return exclude_pattern; +} + +void +netsync_connection_info::Client::set_raw_uri(string const & raw_uri) +{ + parse_uri(raw_uri, uri, origin::user); + + var_key server_include(var_domain("server-include"), + var_name(uri.resource, origin::user)); + var_key server_exclude(var_domain("server-exclude"), + var_name(uri.resource, origin::user)); + + if (db.var_exists(server_include)) + { + var_value pattern_value; + db.get_var(server_include, pattern_value); + include_pattern = globish(pattern_value(), origin::user); + L(FL("loaded default branch include pattern for resource %s: '%s'") + % uri.resource % include_pattern); + + if (db.var_exists(server_exclude)) + { + db.get_var(server_exclude, pattern_value); + exclude_pattern = globish(pattern_value(), origin::user); + L(FL("loaded default branch exclude pattern for resource %s: '%s'") + % uri.resource % exclude_pattern); + } + } +} + +uri_t +netsync_connection_info::Client::get_uri() const +{ + return uri; +} + +void +netsync_connection_info::Client::set_connection_type(netsync_connection_info::conn_type type) +{ + connection_type = type; +} + +netsync_connection_info::conn_type +netsync_connection_info::Client::get_connection_type() const +{ + return connection_type; +} + +bool +netsync_connection_info::Client::get_use_argv() const +{ + return use_argv; +} + +vector +netsync_connection_info::Client::get_argv() const +{ + return argv; +} + +void +netsync_connection_info::Client::maybe_set_argv(lua_hooks & lua) +{ + use_argv = lua.hook_get_netsync_connect_command( + uri, include_pattern, exclude_pattern, + global_sanity.debug_p(), argv + ); +} + +void +netsync_connection_info::Client::ensure_completeness() const +{ + E(!uri.resource.empty(), origin::user, + F("connection resource is empty and no default value could be loaded")); + + E(!include_pattern().empty(), origin::user, + F("branch pattern is empty and no default value could be loaded")); +} + +void +netsync_connection_info::parse_includes_excludes_from_query(string const & query, + vector & includes, + vector & excludes) +{ + includes.clear(); + excludes.clear(); + + char const separator = ','; + char const negate = '-'; + + string::size_type begin = 0; + string::size_type end = query.find(separator); + while (begin < query.size()) + { + std::string item = query.substr(begin, end); + if (end == string::npos) + begin = end; + else + { + begin = end+1; + if (begin < query.size()) + end = query.find(separator, begin); + } + + bool is_exclude = false; + if (item.size() >= 1 && item.at(0) == negate) + { + is_exclude = true; + item.erase(0, 1); + } + if (is_exclude) + excludes.push_back(arg_type(urldecode(item, origin::user), origin::user)); + else + includes.push_back(arg_type(urldecode(item, origin::user), origin::user)); + } +} + +void +netsync_connection_info::setup_default(options const & opts, + database & db, + lua_hooks & lua, + shared_conn_info & info) +{ + info.reset(new netsync_connection_info(db, opts)); + + info->client.ensure_completeness(); + info->client.maybe_set_argv(lua); +} + +void +netsync_connection_info::setup_from_sync_request(options const & opts, + database & db, + lua_hooks & lua, + server_initiated_sync_request const & request, + shared_conn_info & info) +{ + info.reset(new netsync_connection_info(db, opts)); + + info->client.set_raw_uri(request.address); + + bool include_exclude_given = !request.include.empty() || + !request.exclude.empty(); + bool query_exists = !info->client.uri.query.empty(); + + E(!(include_exclude_given && query_exists), origin::user, + F("include / exclude pattern was given both as part of the URL " + "and as a separate argument.")); + + vector includes, excludes; + + if (include_exclude_given) + { + if (!request.include.empty()) + { + includes.push_back(arg_type(request.include, origin::user)); + if (!request.exclude.empty()) + { + excludes.push_back(arg_type(request.exclude, origin::user)); + } + } + } + else + { + parse_includes_excludes_from_query(info->client.uri.query, + includes, + excludes); + } + + info->client.set_include_pattern(includes); + info->client.set_exclude_pattern(excludes); + + info->client.ensure_completeness(); + info->client.maybe_set_argv(lua); +} + +void +netsync_connection_info::setup_from_uri(options const & opts, + database & db, + lua_hooks & lua, + arg_type const & uri, + shared_conn_info & info) +{ + info.reset(new netsync_connection_info(db, opts)); + + info->client.set_raw_uri(uri()); + + vector includes, excludes; + parse_includes_excludes_from_query(info->client.uri.query, + includes, excludes); + + info->client.set_include_pattern(includes); + info->client.set_exclude_pattern(excludes); + + info->client.ensure_completeness(); + info->client.maybe_set_argv(lua); +} + +void +netsync_connection_info::setup_from_server_and_pattern(options const & opts, + database & db, + lua_hooks & lua, + arg_type const & host, + vector const & includes, + vector const & excludes, + shared_conn_info & info) +{ + info.reset(new netsync_connection_info(db, opts)); + + info->client.set_raw_uri("mtn://" + host()); + info->client.set_include_pattern(includes); + info->client.set_exclude_pattern(excludes); + + info->client.ensure_completeness(); + info->client.maybe_set_argv(lua); +} + +void +netsync_connection_info::setup_for_serve(options const & opts, + database & db, + lua_hooks & lua, + shared_conn_info & info) +{ + info.reset(new netsync_connection_info(db, opts)); + info->server.addrs = opts.bind_uris; + + if (opts.use_transport_auth) + { + E(lua.hook_persist_phrase_ok(), origin::user, + F("need permission to store persistent passphrase " + "(see hook persist_phrase_ok())")); + + if (!opts.bind_uris.empty()) + info->client.set_raw_uri((*opts.bind_uris.begin())()); + + info->client.include_pattern = globish("*", origin::internal); + info->client.exclude_pattern = globish("", origin::internal); + } + else if (!opts.bind_stdio) + W(F("The --no-transport-auth option is usually only used " + "in combination with --stdio")); + + info->client.ensure_completeness(); + info->client.maybe_set_argv(lua); +} + // Local Variables: // mode: C++ // fill-column: 76 ============================================================ --- network/connection_info.hh b1cd70d21e53940d512665deea9db7df53207951 +++ network/connection_info.hh 4a6bf8a4ce7bcb2519ad1a993a8ab527e2ab92c7 @@ -11,18 +11,25 @@ #ifndef __CONNECTION_INFO_HH__ #define __CONNECTION_INFO_HH__ -#include -#include +#include "netxx/types.h" +#include "automate_ostream.hh" +#include "database.hh" +#include "options.hh" +#include -#include "automate_ostream.hh" -#include "globish.hh" -#include "uri.hh" -#include "vocab.hh" +struct globish; +struct server_initiated_sync_request; +struct uri_t; +struct netsync_connection_info; + +typedef boost::shared_ptr shared_conn_info; + struct netsync_connection_info { - struct Server + class Server { + public: std::list addrs; } server; enum conn_type @@ -30,26 +37,95 @@ struct netsync_connection_info netsync_connection, automate_connection }; - struct Client + class Client { - globish include_pattern; - globish exclude_pattern; + friend struct netsync_connection_info; + + bool connection_successful; + + bool use_argv; uri_t uri; - utf8 unparsed; std::vector argv; - bool use_argv; + + globish include_pattern; + globish exclude_pattern; + conn_type connection_type; - private: std::istream * input_stream; automate_ostream * output_stream; + + database db; + options opts; + + Client(database & d, options const & o); + ~Client(); + + void set_raw_uri(std::string const & uri); + void set_include_pattern(std::vector const & pat); + void set_exclude_pattern(std::vector const & pat); + void maybe_set_argv(lua_hooks & lua); + void ensure_completeness() const; + public: std::istream & get_input_stream() const; automate_ostream & get_output_stream() const; void set_input_stream(std::istream & is); void set_output_stream(automate_ostream & os); - std::size_t get_port() const; - Client(); + Netxx::port_type get_port() const; + globish get_include_pattern() const; + globish get_exclude_pattern() const; + uri_t get_uri() const; + bool get_use_argv() const; + std::vector get_argv() const; + + void set_connection_type(conn_type type); + conn_type get_connection_type() const; + + void set_connection_successful(); } client; + + static void + setup_default(options const & opts, + database & db, + lua_hooks & lua, + shared_conn_info & info); + + static void + setup_from_sync_request(options const & opts, + database & db, + lua_hooks & lua, + server_initiated_sync_request const & request, + shared_conn_info & info); + + static void + setup_from_uri(options const & opts, + database & db, + lua_hooks & lua, + arg_type const & uri, + shared_conn_info & info); + + static void + setup_from_server_and_pattern(options const & opts, + database & db, + lua_hooks & lua, + arg_type const & host, + std::vector const & includes, + std::vector const & excludes, + shared_conn_info & info); + + static void + setup_for_serve(options const & opts, + database & db, + lua_hooks & lua, + shared_conn_info & info); + +private: + netsync_connection_info(database & d, options const & o); + + static void + parse_includes_excludes_from_query(std::string const & query, + std::vector & includes, + std::vector & excludes); }; #endif ============================================================ --- options_list.hh 8ce5a0d99837f816210671753b564550a563183b +++ options_list.hh 8a5da3dc9c861616ba52de164348285ea902e432 @@ -158,18 +158,17 @@ OPT(min_netsync_version, "min-netsync-ve } #endif -OPT(remote_stdio_host, "remote-stdio-host", - utf8, , +OPT(remote_stdio_host, "remote-stdio-host", arg_type, , gettext_noop("sets the host (and optionally the port) for a " "remote netsync action")) #ifdef option_bodies { - remote_stdio_host = utf8(arg, origin::user); + remote_stdio_host = arg_type(arg, origin::user); } #endif OPT(branch, "branch,b", branch_name, , - gettext_noop("select branch cert for operation")) + gettext_noop("select branch cert for operation")) #ifdef option_bodies { branch = branch_name(arg, origin::user);