quirksOF (progLang).explained (simple)

Creating a Python Wrapper for C++ Classes

This is for Python folks who are not C or C++ experts but need to use a C++ library in their code. This article will show you how to create C++ objects from your Python code and call it's methods as needed.

1. Write an "extern" block for the Classes and methods you want accessible from Python

At the bottom of your C++ code add an extern "C" block.

//Exporting a FlashCards object and two of its methods, getSequence and getCurrentQA.
extern "C" {
    /*
    To expose a C++ class, call its constructor with a *new*, thus returning a
    pointer to the object.
    */

    FlashCards * FlashCards_export (char * fn){ return new FlashCards("civics100.txt"); }

    /*
    To expose a method that returns a C data type, take the pointer to the object it belongs to as the first argument and
    then add its regular arguments. Call the method on that object pointer and return the result.

    Note: the method is called on an object pointer hence the arrow vs the dot
    */

    int FlashCards_getSequence_export(FlashCards* fcs){ return (fcs->getSequence());}

    // To expose a method that returns a C++ data type convert the return type to a C data type. The code below shows
    // how to convert a string return type to a c character pointer return type

    const char *  FlashCards_getCurrentQA_export (FlashCards* fcs) {
        std::string s = fcs->getCurrentQA(); // C++ string
        // string converted to C character pointer
        return std::strcpy (new char [s.length()+1], s.c_str()) ;
    };

Explainer

The code is heavily commented. Here's the lowdown:

1. Each function in the extern block exposes a C++ class constructor or a method to the outside world (in our case to some python code). We will use python's ctypes module to call this exposed classes and methods. ctypes only supports C data types and therefore the functions in this block must receive and return C data types.
2. To expose class constructors we write a function that returns a pointer to the object.
3. To expose methods of an object: we need to take the pointer to the object it belongs to as the first parameter and then add its regular parameters. We then call the method on that object pointer and return its result.
4. If the returned results or the parameters are not C data types, we need to convert them. For example, typically C++ methods will return string data type as opposed to a character pointer, if that is the case we will need the exposing function to convert the string to a character pointer with code like this: std::strcpy (new char [str.length()+1], str.c_str());

Build the library that will be shared across the two languages

This is system dependent. The following has been tested in a Mac.

Compile and create an object file named flashcards.o

g++ -c -fPIC flashcards.cpp -o flashcards.o -std=c++17

Create a shared library fc.so

g++ -shared -W1,-soname,fc.so -o fc.so flashcards.o

That is all from the C++ side. Now lets move to the Python side.

Load the library and create Python wrapper

Add the following to the top of your python code.
lib = ctypes.cdll.LoadLibrary('./fc.so')

Now all the functions that you have exposed in C++ are accessible via the lib variable.
For example we can now refer to the exposed FlashCards_getSequence_export function in python as lib.FlashCards_getSequence_export

The Wrapper Class

In the __init__ of the Python class add argtypes and restypes attributes for every exposed methods' return and argument types. argtypes should be a list (i.e. [])
This is a must.

At last line of the __init__ add self.obj = lib.FlashCards_export(fn) which is the call to the original C++ constructor. This will add the object pointer from C++ to the python code.

For each methods we call the C++ exposed function and return the value. If the C++ function returns a character pointer we need to decode it to make it a Python string.

class FlashCards (object):
    def __init__(self, fn):
        lib.FlashCards_export.argtypes = [ctypes.c_char_p]
                lib.FlashCards_export.restype = ctypes.c_void_p # type for object pointers


        lib.FlashCards_getCurrentQA_export.argtypes = [ctypes.c_void_p]
        lib.FlashCards_getCurrentQA_export.restype = ctypes.c_char_p
        self.obj = lib.FlashCards_export(fn)

    def getCurrentQA (self):
        return str (lib.FlashCards_getCurrentQA_export(self.obj).decode())

That is it !!!. You can now write something like:

civics =FlashCards ("civics100.txt".encode('utf-8'))
civics.getCurrentQA ()

Notes:

Decide which methods and objects you want to be accessible

Unless you are writing a full wrapper of a C++ code base, you do not need to bring in all the code to Python. If you need to do that, handcrafting the wrapper is not a good idea. You should find or write a wrapper generator. In most cases your app will need only a few methods and their classes from the C++ code base. To simplify the handcrafting process you can try not to bring in the element classes of a objects that have containers. For example if you have a FlashCards object that stores and manipulates bunch of Card objects you may have C++ code that does something like the following:

FlashCards fcs ();
fcs.getNext().getQ(); 
// get me the next card object and call its getQ method.

In situations like this instead of bringing in two classes and writing complex wrappers that models interaction between
two class pointers, you should add a FlashCards method like getNextQ () which will execute getNext and then getQ of the getNext and return the Q.

FlashCards fcs ();
fcs.getNextQ(); // get me the Q of the next card object

Now wrap getNextQ;

Calling C code

1. We create a shared object file of our c source code.
2. Then use Python ctypes package load the shared object file and call the function we need to call.

called.c

int call_me (int a, int b)
{
    int s = a * b;
    return s;
}

To create a shared object file type the following

gcc -shared -o c_called.so -fPIC called.c

We will have a shared file name c_called.so in the current diretory

caller.py

from ctypes import cdll
lib_c = cdll.LoadLibrary('./c_called.so') # do not forget the ./

print (lib_c.call_me(3,5))

Calling C++ code

Since ctypes can only talk to C functions, you need to provide those declaring them as extern
"C"

called.cpp

extern "C" {
    int call_me (int p1,int p2)
    {
      int s = p1 * p2;
          return s;
    };
};

To create a shared object file type the following

// g++ -shared -o cpp_called.so -fPIC called.cpp

We will have a shared file name cpp_called.so in the current diretory

caller.py

from ctypes import cdll
lib_cpp = cdll.LoadLibrary('./cpp_called.so') # do not forget the ./

print (lib_cpp.call_me(3,5))

Note

The c and the cpp code above are very similar. We are not showing any C++ object oriented feature. There is a good description in SO for that.

It is quite straight forward but there are several steps that one needs to follow:

Packages you will need to import

import javax.script.ScriptEngineManager;
import javax.script.ScriptEngine;
import javax.script.Invocable;
import javax.script.ScriptException;    

Steps:

1. Create a ScriptEngineManager object
2. Call the object's getEngineByName passing it the string "JavaScript". This will create a script engine object.
3. Call the engine object's eval method and pass it a valid JavaScript string (you can read it in from a file)
4. Cast the engine object as Invocable
5. Call the invocable engine object's invokeFunction and pass it the name of your JavaScript function (not the file) and the list of parameters all seperated by commas.
6. Received the result in an Object variable and then cast it to the expected object.

Example Java

import javax.script.ScriptEngineManager;

import javax.script.ScriptEngine;
import javax.script.Invocable;
import javax.script.ScriptException;

import java.nio.file.Files;
import java.nio.file.Paths;
import java.io.IOException;

class Examples {


  public static String getScript(String fn) {
      String fileContentAsString="";
      try {
         fileContentAsString = new String(Files.readAllBytes(Paths.get(fn)));
      }
      catch (IOException ie) {
        System.err.println (ie.getMessage());
      }
      return fileContentAsString;

  }

  public static String scriptCall () {
    ScriptEngineManager manager = new ScriptEngineManager(); // step 1

    ScriptEngine engine = manager.getEngineByName("JavaScript"); // step 2

    Object result = null;

    try {
      engine.eval(getScript("test.js")); // step 3

      Invocable inv = (Invocable) engine; // step 4

      result = inv.invokeFunction("call_me", "3", "2"); // step 5


    } catch(ScriptException se) {
        System.err.println (se.getMessage());
    }
    catch(NoSuchMethodException nse) {
        System.err.println (nse.getMessage());
    }

    return result.toString(); // step 6 maybe optional 

  }


  public static void main(String[] args) {

    System.out.println (scriptCall());
  }
}

The Javascript file

function call_me (p1,p2){
    var s = p1 * p2
    return s
  }

Perl is a stunningly customizable language. Its original motto "There's more than one way to do it" (TMTOWTDI or TIMTOWTDI, pronounced Tim Toady) makes it a great language for programmers to write very creative code. The flip side of this is that it makes it very hard for someone else to read them. This post thankfully is not about the pros and cons of this motto. Here I show you some code that demonstrates how versatile perl code can be!

In perl you can write a word without any single or double quotes around it and assign it to a variable.

my $bar = TheBar;
say $bar

As long as TheBar is not a declared variable (or file handle or keywords, etc.) perl will assume it is a string (an english word to be precise). These are called barewords in perl lingo. Of course if you think this is not a good idea perl gives you the flexibility to turn the feature on and off as you please with use strict q(subs) and no strict q(subs) respectively.

In perl a quote can be a quote or anything else you want it to be!

Suppose, you want to put three words (e.g. unix commands) in a array. In most languages the code will look something like commands = ['ls', 'pwd', 'who']; In perl there are many ways of doing this:

#you can do the typical of course 

my @commands = ('ls', 'pwd', 'who');
# and you can skip the quotes and do it with barewords as I mentioned above

@commands = (ls, pwd, who); 

# but you can be a cool perl monger and do this 

@commands = qw[ls pwd who];

1. No quotes around the words, less typing.
2. No commas separating the words even less typing!
3. [] instead of () - more common array notation

We can do get it all for the price of that little qw thing.

Enter the q operator

Using the q operator you can decide what your quotes for string literals should be and how do you want perl to interpret the literal.

# use two *s to mark a literal then ask perl to interpret the literal as a single quoted string 

$statement = q *the star is my quote*; # same as $statement = 'the star is my quote';


# just add a q after the q and you have double quotes

$statement = qq *the star is my quote*; #qq for double quote

# double quotes are useful because they allow you to add variables to the literal

$statement = qq * the value is: $command* #will put the value of $command in the literal


# you can even tell perl to interpret literals as shell commands! just add an x after your q

# The following will get the second item from array @commands and execute it as a shell command 

$output = qx ($commands[1]); # execute the item in commands[1] as shell and store the output

And yes we can be both creative and confusing! Can you guess what will be in $output?

@commands = qw[ls pwd who]; 
$output =  qq[ran: $commands[2]\n] . qx($commands[2]);

Aside

TIMTOWTDI motto is opposite of Zen of Python "There should be one — and preferably only one — obvious way to do it".

Python is good with lists. As spreadsheet and relational databases are the two primary workhorses of many information systems, we have an abundance of data in tabular format. Since tables are just collection of lists, python works quite well with tabular data.

Here's some tabular data about cats and dogs.

We can store the data in a two dimensional array. I am skipping the header for this example.

table = [
        ["Foo", "cat", 20, 8],
        ["Bar", "cat", 16, 8],
        ["Boley", "cat", 12,10 ],
        ["Watson", "dog", 70,7 ],
        ["Micha", "dog", 6,3 ],
        ["Tobie", "dog", 2, .5],
    ]

To extract the names (the first entry of every row) and store them in an array we can use python's list comprehension as follows:

# build me a list of row [0] of every row in a table

names = [row[0] for row in table]

We can get fancy and choose some columns and make a list of tuples.

# build me a list of name, age pair from the table

details = [(row[0], row[3]) for row in table]

How about getting all the data of the heaviest animal?

  # isolate the weight data. Its the third column 

  weights=  [row[2] for row in table]
  #find the max (we are assuming there is one singular max)

  max_weight = max (weights) # this is the value not the position

  # Find the row that contains max_weight

  row_num = [row[2] for row in table].index (max_weight)
  # finally get the data

  heavy = table [row_num]
  
  # you can of course write the code in a more "functional" way

  heavy = table[[row[2] for row in table].index(max([ row[2] for row in table]))]

A few myth busting first

1. MongoDB is preinstalled in your workspace. Nope it is not.
2. The sudo apt-get install -y mongodb-org method mentioned all over the Web and in mongodb page as written will work. Nope. It won’t.

What you - need to do

1. Create a fresh node workspace. It is best not to start from a old workspace because cloud9's older workspaces have different software setup.
2. Open a terminal and type cd ~ This command will take you one directory above your workspace. This is actually your true home directory - in the unix file system sense. If you type pwd
   You will see /home/ubuntu We will do most of the work from this directory and not workspace directory as you are used to.
3. From home directory and NOT from workspace directory type the following curl -O https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.2.tgz It will take few seconds to complete and get you back to the prompt. What you are doing is using curl to fetch a zipped version of the mongo db code.
4. Type ls to make sure the mongodb-linux-x86_64-3.4.2.tgz file is in your home directory.
5. Now type tar -zxvf mongodb-linux-x86_64-3.4.2.tgz This unzips and untars the file creating all the directories.
6. Once the command completes type ls again and you will see that the file has created the directories. Make sure you are still in your home (/home/ubuntu) directory.
7. Now we need to modify the .bashrc file. It is a hidden file. So an ls will not show you. Type ls -a (from your home directory :-)) you will see the .bashrc file along with several other hidden files.
8. Type vim .bashrc At the very top, there should be three lines, each preceded by a ‘#’ character. Create a blank line under the third ‘#’ and enter the following: export PATH=/home/ubuntu/mongodb-linux-x86_64-3.4.2/bin:$PATH What you are doing is adding the mongo-db to your path manually
9. Press 'esc' to get into the command mode. Type :wq in the command mode. Do not forget the ‘:’ -- the : switches vim to edit mode and wq says write and quit. If you forget the : you will not be able to get out of text entry mode and your .bashrc file will get wrong data. If that happens hit : again and type q without the the w. You will quit without changes. Then start back again.
10. You are almost there. Type . .bashrc There are two dots. One dot then space then .bashrc (that is the filename)
11. If there are no errors. You are probably doing great
12. Type which mongod you should see the following : /home/ubuntu/mongodb-linux-x86_64-3.4.2/bin/mongod
13. Type mongod & disown This will start the mongo daemon and get you back to the prompt. You may have to hit enter after a few seconds.
14. If you type mongo now - you will be in the database :-)

Issues:

This is a quick and dirty setup.

1. I did not show you how to start the database in a proper way
2. I do not show you how to shut the daemon in a clean fashion. You can do the following:
   • ps - a will show the processes. Write down process number that says mongod
   • kill -9 process num

Before putting real data you must solve these.
This link may be reliable, but this one said mongodb was pre-installed!!

Sorting (placing a collection of items in order) is one of the most common operations in information processing. Arrays are the most common data structure that holds objects we want to sort (collections). So naturally, most modern programming languages provide built-in sort methods to sort arrays. And they are usually very easy and intuitive to use.
In Python for example,

a = sorted ([1, 100,21,9])
print a

will print out the sorted array [1,9,21,100]

In Swift, the code can be even a bit "cute" with emojis!

let 💯 : [Int] = [1,100, 21,9];
print (💯.sorted ())

But Javascript's sort has a mind of its own:

console.log([1,100,21,9].sort())

will printout [ 1, 100, 21, 9 ]
A big oops

Why?

In Javascript, the default sort order i.e. the logic used to determine if an item is greater than, equal to or less than another item uses the ordering of the Unicode strings. The Unicode string '100' is smaller than the Unicode string '21'. Adopting the Unicode as a default sorting order works well with string data but it clearly messes up the sorting of numbers.

The fix

In order for the sort to work correctly with numbers in Javascript we have to override the default ordering by sending a compare function to its built in sort as a parameter. When a compare function is supplied, the sorting order is determined by the return values of the compare function. The compare function has a very specific format and strict requirements.

The requirements

1. It must have two parameters (say a and b)
2. It must return a value that is either less than 0, or greater than 0 or equal to 0
3. It must always return the same value when given a specific pair of elements a and b as its two argument

The override behavior

• if the compare function returns a value that is less than 0, in the overridden Javascript sort, a comes first.
• if the compare function returns a value that is greater than 0, in the overridden Javascript sort, b comes first
• if the compare function returns 0, Javascript should leave a and b unchanged with respect to each other (but sorted with respect to all other specified elements). Note: the ECMAscript standard does not guarantee this behavior, and thus not all browsers respect this.

If you are an experienced Javascript programmer who is familiar with the advanced concepts of Javascript like callback, overloading, function as a first class object etc. the process is actually quite intuitive. But for a newbie it may appear a bit daunting. While we highly recommend you read through this post and follow the links in it, if you are in a hurry to sort your numbers correctly here's the code 😇:

TL;DR: i.e. the code

console.log("% sorted ", [1,100,21,9].sort(compare))
function compare (a, b) {
   if (a < b) {
    return -1;
  }
  if (a > b) {
    return 1;
  }
  return 0;
}

In the code above we are overriding Javascript's default sort order by supplying the compare function. Now instead of using its default sort order to compare and order elements in an array, sort will use the compare function we have provided. Given two numbers, our compare function will return -1, if a is less than b, 1 if a > b and 0 if they are equal.
Therefore by the override behavior described above a will be indexed lower than b if a < b, a will be indexed higher than b if a > b. Thus the sort will work for numbers.

Here is an excellent MDN article explaining the details.

Because we are dealing with numbers, returning the value of a-b has the same effect as spelling out the three cases a > b, a < b and a == b as we have done above. We can make our compare function anonymous and use the a-b pattern to make it a bit more compact like the following:

console.log ([1,100,21,9].sort(function (a,b)  {return a-b})) //ascending

console.log ([1,100,21,9].sort(function (a,b) {return b-a})) //descending


// if you have support for => functions es6

console.log ([1,100,21,9].sort((a,b) => {return a-b})) //ascending

console.log ([1,100,21,9].sort((a,b) => {return b-a})) //descending

The good that comes out of this "pain"

OK, that is a lot to go through to sort a few numbers. We agree! But once you get used to the pattern it will become second nature. Being able to provide a customized compare function to override sort is a very powerful programming tool. We will see one example of it in action in this blog.