talk2stat 0.0.10

Open a bidirectional pipe to R, julia, matlab, or python (etc.) and communicate with it via a socket.

talk2stat

A python package to communicate with statistical software

talk2stat allows programs (such as latextopdf, nodejs) to communicate with statistical packages. Currently, we connect with R, julia, matlab, or python but extending it to other tools like Stata or SAS, is very simple. The package contains two main functions: server() talks to the client program (e.g latextopdf) via a socket (inet), and to the stats package (R, julia, matlab, python) via a bi-directional pipe. The client() function is called by the user (e.g. latextopdf) to establish a connection with the server and pass code to the statistical package, and get the results.

Prerequisites and installation

talk2stat requires Python 3.8 and up. It uses several built-in packages: configparser, sys, os, os.path, unicodedata, re, and socket. Also, it uses the pexpect package to manage the bi-directional pipe to the statistical software. When you install talk2stat, pexpect will also be installed (if not already installed). To install it manually, just use

pip3 install pexpect

And, similarly, to install talk2stat use

pip3 install talk2stat

To install from a local copy,

pip3 install somefolder/talk2stat-0.0.x.tar.gz 

Usage

An invocation of talk2stat is associated with a specific project. For now, it should be used by a single user, although in the future we plan to implement shared sessions. The reason for associating each instance of talk2stat with a single project is to prevent conflicting namespaces, variables, etc. In the current version talk2stat connects with one statistical software per invocation, but we consider a multithreaded version which will allow a user to use the same server to connect with multiple software tools.

In the project directory, create a configuration file for each statistical software you want talk2stat to use. For example, if you plan to use R, create a file called R.config which contains something like the following:

[SERVERCONFIG]
PORT = 65432
DEBUGFILE = Rdebug.txt
PIPETIMEOUT = 300

If you use talk2stat in multiple concurrent projects you must use a different port number for each. Note that currently the IP address of the server is hard-coded to 127.0.0.1. The debug file name (which contains the messages to and from R, in this case) can be the same across projects, since they are stored in different directories. The PIPETIMEOUT is used to specify how long (in seconds) the bidirectional pipe will wait for a response from the statistical package.

To start the server for a specific project we have to specify the location of the configuration file, and the selected language (a project can use more than one statistical package, but as mentioned above, each one uses a different invocation of the server). To start a server which listens to the selected inet port and connects to the selected statistical package (R, julia, matlab, python), use the following:

from talk2stat import server
server(dirname, language)

For example,

from talk2stat import server
server('myproject/', 'R')

The directory must contain a file called R.config. If you plan to use julia, create a file called julia.config and be sure to use a unique port number. Then you can run (a possibly concurrent) server process by running

from talk2stat import server
server('myproject/', 'julia')

The server will continue to run until you stop it (see below how.) This is useful to maintain a continuous session. This is faster and more efficient than invoking R/julia/matlab/python each time a code fragment is executed. The server() function can take an optional third argument called doFork (default = True). If set to False, the server will still continue to run, but it will be blocking and will not return to prompt-mode until the server is stopped. When the server is started by a latex compiler, it must be in non-blocking mode (doFork=True). The blocking mode is appropriate if the server is started manually at a terminal.

Setting the `messages' argument in the server function to True will result in the user session messages to be printed to the screen.

To connect to a running server, we use the client() function, like this:

from talk2stat import client
client(dirname, language, inputfile)

where inputfile contains the code fragment or program to be executed. The output from the server is printed to stdout. The code can also generate files (e.g. graphics, reports).

It is possible to send the code to the client directly, and not through an input file names. The syntax is:

from talk2stat import client
client(dirname, language, "eXline: somecode")

To stop the server, we send the following message (using our previous example) through the client:

client('myproject/','R','QUIT')

Authors

• Haim Bar - Initial work, contributions from HaiYing Wang, Edan Bar

License

MIT