library.txt

UC Library Extensions UnderC comes with a pocket implementation of the standard C++ libraries, wh

    }
 };

I rapidly get bored typing, especially in interactive contexts.   There is a kind of creative laziness which drives programmers to spending late nights working at how to avoid repetitive work.  Here is a function (also from ucri/trace.h) which does all the business of writing a custom trace class for you:

bool add_trace_expr(string fun_pat, string expr)
{
  string cname = tmp_name();
  string cx = "struct " + cname + ": XTrace { ";
  cx += "  void enter(XExecState* xs) { " + expr + "; } };";
  if (uc_exec(cx.c_str())==0) {
    XClass*  pc = uc_global()->lookup_class(cname.c_str());
    XTrace* pt = (XTrace*)pc->create();
    add_trace_to_funs(fun_pat, pt);
    return true;
  } else return false;
}

uc_exec() is used to compile an XTrace-derived class with an overrided enter() method, which carries the payload expression you wish to execute whenever the function(s) are called. add_trace_expr() uses XClass::create() to create a new instance of the class. As promised, it saves a fair amount of typing.  The earlier example of counting how many times string::~string() is called can be expressed in two lines:

;> int k = 0;
;> add_trace_expr("string::__D__","k++");

A very useful function that can be used in trace expressions is dump_fun_args(). For example, say we have a function:

double add(double x, int y, short z)
{ 
  return x+y+z;
}

Then:

;> add_trace_expr("add","dump_fun_args(xs)");
(bool) true
;> add(1,2,3);
x = 1. y = 2 z = 3
(double) 6.

Here is the definition of dump_fun_args():

void dump_fun_args(XExecState* xs)
{
  XFunction* xf = XFunction::from_fb(xs->fb);
  XTList tl;
  XStringList args;
  xf->get_args(&tl,&args);
  string name,s;
  FOR_EACH(name,args) {
     XEntry* pe = xf->lookup_local(name.c_str());
     cout << name << " = ";
     pe->type()->val_as_str(s,pe->data()+xs->bp);
     cout << s << ' ';
  }
  cout << endl;
}

We get a function object from the execute state's function block, and use this to get the actual argument names.  XFunction::lookup_local() allows you to access the otherwise hidden function context, giving the local entries. A pointer to any local variable is by definition some offset plus the base pointer, so we can get the actual values from xs->bp.

dump_fun_args() is just a plain C++ function and can be combined with other calls:

;> void dump_fun(XExecState* xs) {
;1} cout << XFunction::from_fb(xs->fb)->name() << ' ';
;1} }
;; add_trace_expr("add","dump_fun(xs); dump_fun_args(xs)");
(bool) true
;> add(10,20,30);
add x = 10. y = 20 z = 30
(double) 60.

I will now demonstrate a few more voodoo tricks defined in ucri/trace.h:

int sum(int a, int b) 
{
  return a+b;
}

;> sum(10,20);
(int) 30
;> add_trace_expr("sum","RET4(0)");
(bool) true
;> sum(1,2);
(int) 0

The macros RET0 and RET4() force a function to return immediately (sorry RET8() doesn't exist yet for functions returning doubles, but the pattern is clear).  RET0 is easiest to understand; it works by forcing the next instruction to be a RET.

XInstruction ret0_instr = {8,0,0}; // RET

#define RET0 xs->ip = &ret0_instr

There is also CHAIN which allows you to substitute another function altogether. It works by setting the fb and ip directly. You do of course need to make sure that the chained function has the same signature.

;> int product(int x, int y) { return x*y; }
;; add_trace_expr("sum","CHAIN(product)");
(bool) true
;> sum(10,20);
(int) 200
;>

Needless to say, these tricks are very implementation-dependent, which is why I've packaged them up as macros.  (By their nature they cannot be functions)

These are entertaining stunts, but have many uses.  Recently there has been talk of yet another programming paradigm, called Aspect-Oriented Programming (AOP).  It comes from the observation that code is easiest to understand if we try to separate different concerns, or aspects.  For example, logging code is a different aspect of a system and really should not be allowed to intrude into the system's main business.  Rather than having logging code scattered throughout the system, AOP sets up rules so that it is automatically called from all required methods.  Which is precisely what the XTrace facility can give you in UnderC. An another example of AOP that I've seen is when constructing unit tests.  Generally a unit depends on other units, so it can be hard to test individual units.  One traditional approach is to write lots of stub routines, which just return some error status. With XTrace, you can use RET4 to stub out a set of routines automatically.

The UnderC Profiling Facility

UCRI provides precisely one function for profiling execution. This function takes a single argument, which is true if you wish profiling to be switched on, etc, and returns a pointer to an internal 32-bit counter. You can then get an aggregate count of the number of pcode instructions used by your UnderC code, which will usually correspond to the amount of time (unless you have imported functions which take up a significant slice of the execution):

;> int* g_pic = ucri_instruction_counter(true);
;> *g_pic = 0;
(int!) 0
;> double x = 0.0;
;> for(int i = 0; i < 100; i++) x += i;
;> *g_pic;
(int!) 1013
;>

The profiling facility (ucri/profile.h) uses XTrace to attach a special tracing object to every function which we are interested in profiling:

// XProfiler is a custom trace class which is attached to every function;
// it updates the called count and the total cycles used by the function.
class XProfiler: public XTrace {
   int m_kount;
   int m_cycles;
   int m_start;
 public:
   XProfiler() : m_kount(0),m_cycles(0) { }

   int cycles() { return m_cycles; }
   int count()  { return m_kount;  }

   void enter(XExecState* xs)
   {
     m_kount++;
     m_start = *g_pic;
   }

   void leave(XExecState* xs)
   {
     m_cycles += (*g_pic - m_start);
   }
 };

These objects keep an individual count of how many function calls took place, and how many cycles were used in executing this function.

Finding All Function References (ucri/refs.h)

Again, UnderC doesn't give you too much functionality out of the box.  Given an address in a function, XFunction::ip_to_line() will tell you which line number this corresponds to; XFunction::where() will tell you which file the function is in.  There is (currently) no information kept on where all the function references are.  So the strategy is to actually look at the function code.

UnderC provides some assistance; unless the command-line flag -F is used (which attempts to inline one-instruction functions) all functions are explicitly called. The HALT instruction (which usually does breakpoints) acts as a NOP if its operand is not a breakpoint index, and is emitted to make any virtual method calls stand out (since usually VCALL/VCALLX just have a VMT slot as an operand)

;> let pt = new MyTrace();
;> #opt u+
;> pt->enter(0);
0 PUSHI D 34796           // push zero
1 PUSHI D 34792           // push pt
2 LOSS                    // load top of stack onto object stack
3 HALT  D 32935           // NOP + fun block offset
4 VCALLX        D 1       // operand is slot id #1
5 DOS                     // drop object stack
6 RET

This 'fun block offset' needs some explanation.  All instructions are 32-bit, so there's only 22 bits of data possible (8 bits for opcode, 2 bits for address mode). Therefore all 'direct' references to data are actually offsets in a 22-bit data segment. XNTable::offset() will subtract the begining of this segment and give you the offset:

    // find the offset of the function block in the global data segment
    int data = uc_global()->offset(pf->fblock());

The idea is now to go over all the functions and look at each instruction for a DIRECT reference to the function offset.  (Here I use the fact that every function is terminated by an extra instruction with opcode = 0)

void generate(int target_data)
{
    XFunction* fn;
    FOR_EACH(fn,g_fl) {
      // look at the pcode for any direct addressing of the given offset
      XInstruction* pp = fn->pcode();
      if (pp)
       while (pp->opcode != 0) {
         if (pp->rmode == DIRECT && pp->data == target_data)
           g_pos_list.push_back(XPosition(fn,pp));
         pp++;
       }
     }
   }

The rest of the application is just bookkeeping ;).  

An Application: Simple Object Persistence <ucri/persist.h>

Object persistence is a simple strategy for writing and reading a program's object state, which usually has to be done by hand in C++; the strategy is to derive from some base class which defines write() and read() methods, and then each class must overload these methods for their particular case.  Languages with reflection can use the metadata to stream most objects to disk, because they can iterate over all data fields.  So I thought it an interesting experiment to see how this could be implemented with UCRI and UnderC;  in fact, the requirements for UCRI were driven by the particular needs of persistence.

Personally I have mixed feelings about persistence, coming mostly from practical experience with the MFC framework, which is particularly nasty in that it generates an opaque binary file format.  Although traditional persistence makes sense from the point of view of object-orientation (each object becomes responsible for its external representation) the result is a file with structure reflecting the internal object hierarchy, which breaks encapsulation of the system as a whole.  So you cannot change the internal arrangement of objects without breaking the file format.  And this is no theoretical problem, as I can attest from years of maintaining MFC systems. So we'll avoid binary formats in this application.

Some interesting constraints on C++ coding style emerge from this experiment.  First, you have to organize your objects so that there is an object which defines the root of a hierarchical tree of objects. So when the root object is told to stream itself out to disk it will result in all objects being streamed out.  (It's probably possible to relax this requirement by asking all objects found in namespaces to stream out, but a root object is a useful disciple anyhow).

Second, pointers to objects must be used in a disciplined way; I have to assume that if I see a pointer, it refers to a single object and not a dynamically allocated array of objects.  (Regular arrays are fine because they have a definite size) This is not a bad restriction, since we should be using some container like a list or a vector anyway.  Although the implementation of vector usually involves precisely such dynamic arrays, so the approach I've taken here is to explicitly handle vector and list.

Third, classes must have a default constructor.  I have to create the objects dynamically when streaming in objects, which is done using XClass::create().

PERSIST.H is probably the hardest three hundred lines of code I've ever written, and so don't feel obliged to understand every detail at first.  (If the gods be kind and I have time to develop this further, it will become yet another UCRI utility that can be used without too anxiety).  A useful place to start is OutStreamer::stream(XEntry* xe), which writes out an ASCII representation of the entry 'xe'.

line 192:  xe->ptr(m_base). You've seen XEntry::ptr() with its default NULL argument - in general a base pointer is supplied, if you don't want to use the global address space.
line 198:  char* is treated specially, because it's one pointer-to-value which _by convention_ carries its own length information.  Yes, sometimes a char* may legitimately contain null characters, in which case wrap it up as a string (and then handle string as a special case)
line 203:  Array entries contain size information, so they're cool. The trick is to use XEntry::base_entry() to generate an entry for the first element of the array, and then mosey along the array, streaming each element in turn, moving to the next element by incrementing the entry data using the element size:

void Streamer::stream_array(XEntry* xe)
{
  XEntry* be = xe->base_entry(); // bogus entry for any element
  int sz = be->type()->size();   // size of element type
  int n  = xe->size();           // number of elements
  be->set_data(xe->data());
  for(int i = 0; i < n; i++) {
    stream(be);                     // stream out element
    be->set_data(be->data() + sz);  // and move to the next
  }
}

line 205: Plain scalar values can be converted to a string representation using XEntry::val_as_str().

line 209: Pointers or references to objects are a special case. The pointer value is output, followed by the _actual dynamic type_ of the object.  This is a crucial point; a pointer may be declared to be A*, whereas it's actually pointing to some object of type C which is derived from A.  We should only stream unique objects out once, so I keep a map of objects which have been processed.

line 220: Only list<> or vector<> at this point! 

line 224: Streamer::stream_object() is used to stream out an object's fields. It's a classic use of reflection - find all the non-static member variables and stream each one of them out in turn.

void Streamer::stream_object(XClass* pc, void *ptr)
{
  XEntry* ce;
  void *old_base = m_base;
  XEntries xlist;
  XEntries::iterator xli;
  set_base(ptr);
  pc->get_variables(xlist,NON_STATIC | FIELDS);
  for(xli = xlist.begin(); xli != xlist.end(); ++xli) {
       ce = *xli;
       stream(ce);
  }
  set_base(old_base);
}

The tricky bit is how to handle containers. Streaming out elements is itself a straightforward function template:

template <class C>
  void stream_elements(const C& ls, Streamer& out, XEntry* xe)
  {
     C::iterator it = ls.begin(), iend = ls.end();
     for(; it != iend; ++it) {
        xe->set_ptr(& *it);    // entry now points to element
        out.stream(xe);        // which can be streamed out/in
     }
  }

But we don't know what the type is at compile-time, so this function has to be _dynamically_ instantiated.  (Ditto for a little template size_of() to find a container's size).  

The object model supported by PERSIST.H is still too limited to be generally useful. For instance, what happens to other template classes?  UCRI object persistence is obviously not portable, and that's an issue for me because I try not to get trapped into using a non-portable dialect for larger programs.  An interesting project would be to actually generate   the C++ code for persistence explicitly and weave it into the source.