Creating a Shared Library Bundle with Package Compiler

07/29/2021, 7:30 PM7:40 PM UTC
Blue

Abstract:

PackageCompiler.jl has become the de facto method for creating standalone Julia applications. In this talk, we will demonstrate the use of PackageCompiler.jl to produce shared library bundles. This functionality was added recently and allows the easy creation of location-independent dynamic libraries which can be linked to and called from C, C++, Rust, or other languages which can link to and use C libraries.

Description:

Julia has been touted as a great solution to the two-language problem (and it is). But for many, interacting with code in other languages is a necessity.

Numerous packages exist which aid interoperability with other languages, including C (Clang.jl), C++ (CxxWrap.jl), Java (JavaCall.jl), Matlab (Matlab.jl / Mex.jl), Python (PyCall.jl / pyjulia), R (RCall.jl / JuliaCall), Mathematica (MathLink.jl), and rust (jlrs).

Many of these packages focus on calling out to code in other languages from Julia, but there is also support for calling Julia code from other languages, especially for those that have the ability to call C functions, and that is what we will focus on here.

The Julia manual has a full section on Embedding Julia. Until now, this has been the standard way to embed and call Julia from other languages. Using the ideas here, along with custom Julia sysimage generation with PackageCompiler.jl, one of us created a proof-of-concept repository for creating a shared library from Julia code for C or other languages (https://github.com/simonbyrne/libcg).

One downside of this work is that the library was not easy to relocate--it contained hard-coded paths to the Julia runtime. We wanted the ability to create a relocatable shared library.

PackageCompiler.jl already allowed the creation of “apps”--bundles of files, including an executable--which could be relocated and moved to other machines (with some minor caveats). We extended this functionality to create relocatable shared libraries with a create_library function.

The actual act of creating a shared library with PackageCompiler.jl is very much like creating an “app”, and has a very similar output--a bundle of directories which include the shared library and enough of the Julia runtime to run. This bundle can be zipped or tarred up, sent to other computers, and installed in any location that a linker can find it. The user has the option of setting the library version (on Mac and Linux), and can include C header files for the Julia functions she has exported in the shared library.

For this talk, we will give a brief overview of the create_library functionality, discuss situations in which it might be used, show how to use it, and discuss its limitations.

Platinum sponsors

Julia Computing

Gold sponsors

Relational AI

Silver sponsors

Invenia LabsConningPumas AIQuEra Computing Inc.King Abdullah University of Science and TechnologyDataChef.coJeffrey Sarnoff

Media partners

Packt Publication

Fiscal Sponsor

NumFOCUS