Re: [Help-smalltalk] the test test - a more detailed analysis of image contents

[Stefan Schmiedl]

I really should make a blog post out of this ... heck it's enough
for another mini-series :-)

Executive Summary:

There are MANY objects in the image related to Iliad.
How do we get rid of them?


On Fri, 17 Jul 2009 15:09:21 +0200
Stefan Schmiedl <address@hidden> wrote:

> > What is the actual size of the images?  
> 
> address@hidden testdrive $ ls -l ../ot.im *.im
> -rw-r--r-- 1 stefan stefan  6234160 16. Jul 17:14 ../ot.im   <- "clean image"
> -r--r--r-- 1 stefan stefan 76702080 16. Jul 09:30 ot6a.im    <- "used image"
> -r--r--r-- 1 stefan stefan 92434208 16. Jul 10:35 ot6b.im    <- "special case"
> -r--r--r-- 1 stefan stefan 74360512 16. Jul 11:24 ot6c.im    <- "used image"
> -r--r--r-- 1 stefan stefan 79733448 16. Jul 11:59 ot6d.im    <- "used image"
> -r--r--r-- 1 stefan stefan 79306448 16. Jul 12:33 ot6e.im    <- "used image"

ot.im is the preloaded image, my OnlineTester package, depending
on Iliad. The other images were saved via 
  gst-remote --eval="ObjectMemory snapshot"
after finishing the test session.

I ran a bit of code on each of these images:

Object subclassesDo: [ :c | |s|
  s := c allInstances size.
  s > 0 ifTrue: [ 
    c name print. 
    ': ' display. 
    s printNl
  ]
]

Then I realized, that I was missing all the interesting classes,
but manually comparing allSubclassesDo output is really too much...
so let's gain some hacking points.

Collect the "clean" state in ot.im:
  usage := IdentityDictionary new.
  Object allSubclassesDo: [ :cls | |count|
    count := cls allInstances size.
    usage at: cls put: count
  ].
  s := FileStream fopen: 'counts.dump' mode: #write.
  [ ObjectDumper dump: usage to: s ] ensure: [ s close ]

The dictionary usage contains 1237 items and we can now
compare it to the "used" state in the other images:
  s := FileStream fopen: 'counts.dump' mode: #read.
  [ usage := ObjectDumper loadFrom: s ] ensure: [ s close ]
  Object allSubclassesDo: [ :cls | |count orig|
    count := cls allInstances size.
    orig := usage at: cls ifAbsent: [ 0 ].
    ( count > 1 and: [count > (orig * 2)] ) ifTrue: [ 
      cls display.
      ': ' display.
      orig print.
      ' -> ' display.
      count printNl.
    ]
  ].

ot6a.im has run with an error in the application code, so
the data might be a little bit off, let's look at some
impressive changes in ot6b.im. I've grouped them into some
chunks of related (in my point of view) classes.


"My" object counts match expections:
  OnlineTester.OTApplication: 0 -> 27
  OnlineTester.OTAufgabe: 0 -> 13
  OnlineTester.OTFrage: 0 -> 95
  OnlineTester.OTRegisterWidget: 0 -> 27
  OnlineTester.OTTestWidget: 0 -> 27
  OnlineTester.OTAufgabeWidget: 0 -> 351     = 13 * 27
  OnlineTester.OTFrageWidget: 0 -> 2565      = 95 * 27 (I guess)


Swazoo and Socket objects don't seem to be out of range, either,
considering the Process count.
  Process: 5 -> 258
  Swazoo.ContentTypeField: 0 -> 246
  Swazoo.GenericHeaderField: 0 -> 2704
  Swazoo.HTTPAcceptField: 0 -> 246
  Swazoo.HTTPConnection: 0 -> 246
  Swazoo.HTTPConnectionField: 0 -> 492
  Swazoo.HTTPContentLengthField: 0 -> 246
  Swazoo.HTTPDateField: 0 -> 246
  Swazoo.HTTPGet: 0 -> 246
  Swazoo.HTTPHeaders: 0 -> 492
  Swazoo.HTTPHostField: 0 -> 246
  Swazoo.HTTPRefererField: 0 -> 201
  Swazoo.HTTPRequestLine: 0 -> 246
  Swazoo.HTTPResponse: 0 -> 246
  Swazoo.HTTPServerField: 0 -> 246
  Swazoo.HTTPSetCookieField: 0 -> 206
  Swazoo.HTTPUserAgentField: 0 -> 246
  Swazoo.SwazooBuffer: 0 -> 246
  Swazoo.SwazooSocket: 0 -> 248
  Swazoo.SwazooStream: 0 -> 246
  Swazoo.SwazooTask: 0 -> 246
  Swazoo.SwazooURI: 0 -> 447
  Swazoo.URIResolution: 0 -> 246
  Sockets.IPAddress: 0 -> 500
  Sockets.ReadBuffer: 0 -> 247
  Sockets.ServerSocket: 0 -> 3
  Sockets.StreamSocket: 0 -> 246
  Sockets.TCPSocketImpl: 0 -> 251
  SpIPAddress: 0 -> 2
  SpSocket: 0 -> 248
  SpTimestamp: 0 -> 492


Iliad object counts seem to be a bit high, except for Session,
and they stay that way, even after a garbage collection.
  Iliad.Session: 0 -> 28
  Iliad.Action: 0 -> 16618
  Iliad.BreakElement: 0 -> 16613
  Iliad.Decorator: 0 -> 2999
  Iliad.DivElement: 0 -> 15498
  Iliad.FormElement: 0 -> 5166
  Iliad.Id: 0 -> 3054
  Iliad.InputElement: 0 -> 5166
  Iliad.LabelElement: 0 -> 16613
  Iliad.RadioButtonElement: 0 -> 16613
  Iliad.RawHtmlElement: 0 -> 26945
  Iliad.SpanElement: 0 -> 5166

I guess the following counts are closely related to
the Iliad objects:
  BlockClosure: 1254 -> 19541
  BlockContext: 304 -> 23255
  String: 14635 -> 95456



The FileStream count shows that my apache proxy did not work
as expected. Most of those are pointing to static files I had
tried to keep out of swazoo's hair.
  FileStream: 3 -> 33


Some things have gone wrong during usage:
  Kernel.CoreException: 7 -> 19

Let's look at them:
  Kernel.CoreException allInstancesDo: [ :e |
    e signalClass printNl
  ]
    Error
    Exception
    Halt
    Iliad.CurrentContext
    Iliad.DispatchError
    Iliad.ResponseNotification
    Kernel.CoreException
    MessageNotUnderstood
    Notification
    Signal
    SpAbstractError
    SpError
    Swazoo.HTTPException
    Swazoo.SwazooStreamNoDataError
    SystemExceptions.FileError
    SystemExceptions.PrimitiveFailed
    SystemExceptions.ProcessBeingTerminated
    SystemExceptions.UnhandledException
    SystemExceptions.UserInterrupt
    SystemExceptions.VMError



The most shocking increases are found in the collection
hierarchy:

  IdentityDictionary: 17 -> 48
    let's count the sizes
      Bag(0:1 1:30 251:1 3:2 2:1 6:1 14:1 13:1 17:1 23:1 22:1 )
        What do I need 30 IdentityDictionaries with 1 item for?
          OnlineTester.OTApplication->an OTApplication
        ah ... must come from iliad sessions

  WeakArray: 3 -> 993
    hmm... they're gone now ... weaklings, afraid of being
    examined.

  HomedAssociation: 3 -> 548
    note to self: look up what that is.
    map temporary stuff to nil


  Association: 524 -> 178658
  Dictionary: 10 -> 108373
    the former are stored in the latter and are almost
    completely iliad stuff

  OrderedCollection: 60 -> 109188
    let's count sizes again
      Bag(0:70960 32:3 1:27178 97:1 3:5211 2:160 64:1 
        7:85 4:57 6:57 5:3 288:1 135:1 14:1117 8:199 
        13:30 9:28 11:4052 10:31 43:2 12:3 576:1 28:1 
        17:1 92:1 18:1 3246:1 20:1 2304:1 1152:1 )
      ooh ... 70k empty OC's 
      the 27k single element OC's have either a RawHtmlElement
      or a cookie
      the 5k 3-element OC's are either the mime-type collection
        ('application/json' ' text/javascript' ' */*' )
      or
        (a DivElement a FormElement a DivElement )
      NB the singel OC with 3246 elements contains either
      classes or (quite often) nil ... what's that?