Re: optimized SHA1 (was Re: [Monotone-devel] net.venge.monotone.experiment.performance)

[Jack Lloyd]

On Sat, Aug 05, 2006 at 12:40:17AM -0700, Nathaniel Smith wrote:

> We should also get Jack Lloyd involved in this discussion -- I think
> he reads the list, but I've CC'ed him just in case.

I am, though thanks for the CC, as I often don't read everything.

> In particular, is it possible to add such cores to Botan externally,
> I suppose using the "engine" support?  (I don't see any
> documentation on the web site for what engines can do, or how to
> write one.)  And what are your thoughts on adding such asm cores to
> Botan proper?

The engine stuff is very poorly documented right now, but it's (at
least superficially) like the engine code in OpenSSL - you can plug in
new backend implementations of algorithms. The most immediately useful
example is going to be the OpenSSL engine (in modules/eng_ossl in the
main source tree).

There are two basic ways I can imagine handing it - derive a new
object from Botan's SHA_160 class and just override the hash()
function (which implements the core transform), leaving the default
padding/buffering code, or if you think those virtual function calls
are causing a problem (I'd be skeptical, but who knows) and/or want to
implement everything in asm, you can implement a completely distinct
HashFunction object and ignore the default implementation
completely. The engine itself would look like (completely untested
code):

class Monotone_Engine : public Botan::Engine
   {
   public:
      Botan::HashFunction* find_hash(const std::string& name) const
         {
         if(name == "SHA-160")
            return new Monotones_SuperFast_AsmOptimized_SHA1();
         return 0;
         }
   };

In 1.5 you'd add this to the library with a call like
  global_state().add_engine(new Monotone_Engine);
Or in 1.4
  Engine_Core::add_engine(new Monotone_Engine);

In cases where find_hash returned NULL, the library would simply
continue on until it found an Engine that returned an object
(typically, the Default_Engine, which has all the normal builtin
versions).

BTW, Botan is being developed under Monotone now, server is at
randombit.net, branch is net.randombit.botan. Which should make
syncs a little more convenient for you guys, I guess.

I've also updated the wiki page a bit to provide some info re Botan
about including asm code and zero copy calls.

Jack