No LoadMap ( ) function in leadwerks.dll?

[Shaping]

Did you look at SWIG, mentioned on the previous page? ( http://www.swig.org/ )

 

Yes, briefly. But why use it? I have an ExternalLibrary building facility in my environ. I feed it a C header file and it creates the ExternalInterface subclass and its methods (one per function). This is the programming environment's model of a DLL and its entry points.

 

It wraps C++ classes from languages with an FFI (sorry to burst the "cannot be done" bubble). If you can get it outputting something your language can understand you shouldn't need a C-style DLL at all.

 

That was my point. The FFI of the environment is a C-based one, not C++-based. There is no point in modeling C++ objects on the foreign heap when you can model Smalltalk objects on the native heap, and have just as much speed and better debugging. You get the speed improvements by outboarding compute-intensive methods as C-functions. Wrapping C++ won't get me speed. SWIG looks like a way to use a framework that for whatever reason is not or cannot be modeled in a flat way as functions. SWIG looks like a convenience that has value when the equivalent flat model does not exist. I thought we were working on a Leadwerks flat version, too, which I prefer.

 

Have you benchmarked a C++ wrapping?

 

Leadwerks.h requires a small amount of tidying in order to work with it, but the main API does export correctly once you've put the right guards in place and so on.

 

Guards?

 

If your language isn't supported directly, SWIG can dump a full parse tree in Lisp format which you should be able to pass through a second layer to process into declarations for the thing you want to use (or maybe one of the others will be close enough; I found the CFFI backend gave all of the necessary data in a much smaller file).

 

Smalltalk does not seem to be supported directly. Are you saying that SWIG produces a description of C++ classes that can be parsed to produce equivalent flat functions?

 

 

e.g. put this together in a couple of minutes: https://dl.dropboxusercontent.com/u/46635275/SWIGtest.zip

(Leadwerks_G.h is a preprocessed version,

[/quote/

 

Where did this come from?

 

only contains limited declarations for export, skipping the irrelevant and the difficult; Leadwerks_wrap.cxx is the generated C-style library; Leadwerks.i is the interface definition that passes the original header to SWIG; the Lisp files are the generated CFFI output.)

 

Where is object state maintained? For productive use of one's environ, this must happen in the environment's process space--on the native heap in the case of Smalltalk. If SWIG uses the foreign (C-based) heap, then are the object's instance variables unwrapped and moved to a more global scope? Or do we have no state at all and just C functions? Broadly, the question is: What do we do about maintaining state (object state) across multiple invocations of a function? If the client (the environment) is creating all the objects and hence all the state, then we must pass those variables each time we call a function.

[Yasha]

Guards = #if 0 ... #endif around things you don't need.

 

Leadwerks_G.h is my own doing: after sticking in a bunch of said guards to get rid of unwanted things like the standard libraries, run it through gcc -E to include all files and expand all macros, creating a "flat" Leadwerks header. This is not necessary to use SWIG in the general case (SWIG is built on GCC anyway, it can do preprocessor stuff perfectly well), but it gave a clearer idea of what was going on, I thought.

 

Are you saying that SWIG produces a description of C++ classes that can be parsed to produce equivalent flat functions?

 

SWIG produces C-style code like this:

 

EXPORT void _wrap_Entity_Translate__SWIG_1 (Leadwerks::Entity *larg1, float larg2, float larg3, float larg4) {
 Leadwerks::Entity *arg1 = (Leadwerks::Entity *) 0 ;
 float arg2 ;
 float arg3 ;
 float arg4 ;

 arg1 = larg1;
 arg2 = larg2;
 arg3 = larg3;
 arg4 = larg4;
 try {
   {
  (arg1)->Translate(arg2,arg3,arg4);
   }

 } catch (...) {

 }
}

 

(Longwinded to read but will be identical to a one-liner after optimisation.) That's what is found in Leadwerks_wrap.cxx: C-style wrappers for C++ calls, creating a completely flat API. I assume this also answers the question about heap usage (which I otherwise don't follow, sorry).

 

It also produces FFI declarations for the requested language to make these C-style wrapper functions available. How much more wrapping takes place at that level depends on how that language's FFI works (the CFFI output is nice primarily because it consists entirely of easily-parsed type declarations - I assume a Common Lisp interpreter will build glue out of these itself - rather than actual C glue code to do the dynamic type checking like it outputs for some other targets; this means you can extract the types for other uses).

[Shaping]

Guards = #if 0 ... #endif around things you don't need.

 

Leadwerks_G.h is my own doing: after sticking in a bunch of said guards to get rid of unwanted things like the standard libraries, run it through gcc -E to include all files and expand all macros, creating a "flat" Leadwerks header. This is not necessary to use SWIG in the general case (SWIG is built on GCC anyway, it can do preprocessor stuff perfectly well), but it gave a clearer idea of what was going on, I thought.

 

 

 

SWIG produces C-style code like this:

 

EXPORT void _wrap_Entity_Translate__SWIG_1 (Leadwerks::Entity *larg1, float larg2, float larg3, float larg4) {
Leadwerks::Entity *arg1 = (Leadwerks::Entity *) 0 ;
float arg2 ;
float arg3 ;
float arg4 ;

arg1 = larg1;
arg2 = larg2;
arg3 = larg3;
arg4 = larg4;
try {
{
(arg1)->Translate(arg2,arg3,arg4);
}

} catch (...) {

}
}

 

(Longwinded to read but will be identical to a one-liner after optimisation.) That's what is found in Leadwerks_wrap.cxx: C-style wrappers for C++ calls, creating a completely flat API. I assume this also answers the question about heap usage (which I otherwise don't follow, sorry).

 

It also produces FFI declarations for the requested language to make these C-style wrapper functions available. How much more wrapping takes place at that level depends on how that language's FFI works (the CFFI output is nice primarily because it consists entirely of easily-parsed type declarations - I assume a Common Lisp interpreter will build glue out of these itself - rather than actual C glue code to do the dynamic type checking like it outputs for some other targets; this means you can extract the types for other uses).

 

Okay, I see the pattern: SWIG uses always in its flattening functions a list of parameters with a head and a tail. The first parameter (the head) is always the object whose method is called. The remaining parameters are the parameters in that method. The name of the function on the object is the middle of the flat function name. The flat function name here is _wrap_Entity_Translate__SWIG_1. The method name is Translate. I see that the _wrap_Entity_ and SWIG_1 are added automatically by the SWIG wrapping process, and suppose that the user of SWIG can determine what these pre- and post-fixed strings will be.

 

Because the object pointer is the first parameter and is passed-in from the client program, the Smalltalk or the C# is obviously creating and holding the object there either on the heap or transiently on the stack, as methods come in and out of scope.

 

However, because the SWIG-created flat function calls a method on a C++ object, I don't see how to use that function. Where are these C++ objects being created? This was my question above. Where is the state created and maintained, in a C++ environment or in the client Smalltalk or C# environment. I think we both want to do this in Smalltalk/C#, not in C++. So, I don't see the utility of SWIG in this case.

 

My Smalltalk environ has a foreign C-heap but this is not a C++ heap. The foreign heap gives no way to attach procedures to data on the C- heap, as far as I know (I'll check that...). In other words, I don't think I can emulate a C++ object in my environ's C-heap, and that does not seem productive anyway. It could probably be engineered, but the idea is a bit repulsive, because the situation is already awkward, with the head and tail of the flat-function's parameter-list being the object and arguments, respectively. This already is a shape change in function structure.

 

Ideally, I want DLL functions with only the tail parameters (method parameters). (Note that outboarding a class method as a DLL function is not desirable unless the function is very compute-intensive--say lots of matrix math--and the time to marshal the result to and from the C function is tiny compared to the time saved doing the computation outboard with unboxed floats and doubles.) I suppose I can still use the auto-generated SWIG functions, and then just go into those and manually edit off the first parameter, but then I must provide the source code for each of those methods. If we have that in the C++ that comes with our standard license, I'm okay with doing that manually. Yes, it's a lot of work. Do we have all the code to do that? Does anyone know? Josh said that the DLL does not cover all functional areas. Does the C++ source cover all functional areas? If any of those provided C++ methods call or instantiate deeper undocumented/unprovided functions or classes, then I'll be stuck, and will not be able to finish implemented the DLL methods. See the problem? Am I missing something?

 

Can anyone answer my question about the threading model for the graphics and physics? These two are likely running on separate threads. I'll assume also that loads of models, textures, sounds, fonts--assets--are also done on separate threads. I'm assuming that we have a synchronized shared queue that lets the rendering thread talk to the physics thread. I think I can handle three threads in the Smalltalk version of this, for starters. But for the time being, to get something working, I just want to focus on one thread for everything.

 

Do we have a simple tutorial that can draw a finely tesselated sphere? This is where I'd like to start: One good-looking sphere, ambient light, two directed lights, and a short blurb of text rendered with a flat font parallel to and at or near the near clipping plane of the frustum, all running on one thread in a dynamic environment. I have a high-frequency, icosahedral-sphere vertex-generating method in Smalltalk. I've tested it already. I probably want to save that vertex set down as a model, and load it. (Do we have a tut on how to do that?) I'll like to see this simple scene work as a Leadworks 3.1 scene, and build up from there.

[Yasha]

Because the object pointer is the first parameter and is passed-in from the client program, the Smalltalk or the C# is obviously creating and holding the object there either on the heap or transiently on the stack, as methods come in and out of scope.

 

However, because the SWIG-created flat function calls a method on a C++ object, I don't see how to use that function. Where are these C++ objects being created? This was my question above. Where is the state created and maintained, in a C++ environment or in the client Smalltalk or C# environment. I think we both want to do this in Smalltalk/C#, not in C++. So, I don't see the utility of SWIG in this case.

 

My Smalltalk environ has a foreign C-heap but this is not a C++ heap. The foreign heap gives no way to attach procedures to data on the C- heap, as far as I know (I'll check that...). In other words, I don't think I can emulate a C++ object in my environ's C-heap, and that does not seem productive anyway. It could probably be engineered, but the idea is a bit repulsive, because the situation is already awkward, with the head and tail of the flat-function's parameter-list being the object and arguments, respectively. This already is a shape change in function structure.

 

Ideally, I want DLL functions with only the tail parameters (method parameters). (Note that outboarding a class method as a DLL function is not desirable unless the function is very compute-intensive--say lots of matrix math--and the time to marshal the result to and from the C function is tiny compared to the time saved doing the computation outboard with unboxed floats and doubles.) I suppose I can still use the auto-generated SWIG functions, and then just go into those and manually edit off the first parameter, but then I must provide the source code for each of those methods.

 

The object is created on the C++ heap. Different wrapper functions return C pointers to C++ objects (how these are translated into values of your language depends on how the FFI handles returned C values; a high-efficiency FFI probably wouldn't create anything on the client language's heap at all). So when you call a wrapped method such as the one in the example, you're just passing back to C++ a pointer that originated from C++ in the first place.

 

DLL functions can't just have the tail parameters. This doesn't make any sense at all! How would the method operate on an object if it never receives any object reference? (C functions operate at a lower level than what you might be used to from a language like Smalltalk: functions are just static vectors of machine code, they cannot retain references to nonglobal values - the object must be passed in, either as a parameter, or through the syntactic sugar that is C++ method call syntax.)

 

Functions such as the above - with the C++ object pointer as the first parameter - are exactly what you need for a language binding, to remotely manipulate objects located entirely within the C++ layer.

[Shaping]

The object is created on the C++ heap. Different wrapper functions return C pointers to C++ objects (how these are translated into values of your language depends on how the FFI handles returned C values; a high-efficiency FFI probably wouldn't create anything on the client language's heap at all). So when you call a wrapped method such as the one in the example, you're just passing back to C++ a pointer that originated from C++ in the first place.

 

DLL functions can't just have the tail parameters. This doesn't make any sense at all! How would the method operate on an object if it never receives any object reference?

 

As mentioned, you would need to pass the object's instance variables by value into the DLL function. You would get one one or more values back from the DLL call and update your client-side object ivars.

 

(C functions operate at a lower level than what you might be used to from a language like Smalltalk: functions are just static vectors of machine code, they cannot retain references to nonglobal values - the object must be passed in,

 

The object or in this scenario the relevant parts of it (its ivars), as needed by the current function.

 

either as a parameter, or through the syntactic sugar that is C++ method call syntax.)

 

Functions such as the above - with the C++ object pointer as the first parameter - are exactly what you need for a language binding, to remotely manipulate objects located entirely within the C++ layer.

 

Yes, but I don't have C++ objects to begin with. I can get close enough, though. I have C data structures on the client's foreign heap that can function as the data part of the object, but not as the function part. That's okay, as long as all the functions are in the DLL, and as long as the shape of that data is as it would be in a C++ object, including alignment and endianness.

 

I just need some extra client-side code to place the Smalltalk object on the foreign heap as a C object. I wasn't thinking about doing that earlier. I wanted to emulate the entire C++ object, but that isn't necessary. So in summary you can pass the parts (the ivars of the object) or pass the whole object. The second creates easier coding, but requires some preparation and will produce a speed hit, because all the St heap objects must be mirrored on the foreign heap. (I'll get back to this any day now and test the idea, but I need first to work some of the simple C++ demos and tuts to see how the rendering loop works.) Thanks for shaking me out of my fog. I thought I was missing something obvious. And to really take advantage of the SWIG, I must pass the whole object, which would be my preference.

 

I've done this sort of thing in the past, where I've taken a set of C++ classes, converted them to matching Smalltalk classes, and then from within the Smalltalk client, at run-time dynamically externalized these objects on the client's foreign C heap, so that some DLL can use them. It works pretty well, but there are start-up and shut-down phases during which the new C objects must be created by the client and then later freed, respectively. It's expensive, and is practical only at low frequencies. Some of our C++ objects (and therefore these matching C objects) must come and go very quickly in Leadwerks 3.0. So, depending on the speed of the C-object creation/release flux, it may or may not be practical. The St foreign heap thrashing will ruin run-time efficiency. That is why I was more inclined initially toward just tail-argument DLL functions and a manual (slow and careful) conversion all needed C++ code.

 

Are the choices and problems making more sense now?