In technical computing, getting data into and out of your code can be a pain. Data comes in all shapes, sizes and formats, with many different locations and storage access mechanisms.
DataSets.jl is a new package for describing data declaratively and mapping it neatly into your programs. We aim to make your code portable between data environments and remove the cruft of local paths and data access wrappers which litter technical analysis code.
The Scalable Integrated Infrastructure Planning (SIIP) initiative at NREL has developed a set of high-performance power system simulation capabilities with PowerSystems.jl and PowerSimulations.jl. This talk will demonstrate these capabilities with interactive examples using large realistic datasets, and provide theoretical background for software design choices.
The goal of this session is to gather people interested in streaming their programming sessions. Streaming code is a particular exercise, we want to foster the exchange of best practices, tips and thoughts. In particular, we would like to see to which extent Julia streamers have managed to reach an audience beyond the Julia community, used streams as a teaching medium and how we can improve the formats to make it more accessible to newcomers. The BoF is not limited to streamers, people watching
Demonstrate SmartTensors (http://tensors.lanl.gov; https://github.com/SmartTensors): a toolbox for unsupervised machine learning based on matrix/tensor factorization constrained by penalties enforcing robustness and interpretability (e.g., nonnegativity; physics and mathematical constraints; etc.). SmartTensors has been applied to analyze diverse datasets related to a wide range of problems: from COVID-19 to wildfires and climate.
Systems Biology Markup Language (SBML) and CellML are extensible markup languages (XML) widely used throughout the biological modeling community. In this talk we showcase new packages (SBML.jl and CellMLToolkit.jl) for importing models from these languages to the ModelingToolkit.jl format for the full suite of SciML tools to simulate and analyze!
We develop multiscale models that couple cell-cell communication with cell-internal gene regulatory network dynamics to study cell fate decision-making from a dynamical systems perspective. In JuliaLang, we model cell-cell communication as a Poisson process, and cell-internal dynamics with nonlinear ordinary differential equations, taking advantage of the power of DifferentialEquations.jl. We show that subtle changes in cell-cell communication lead to dramatic changes in cell fate distributions.
In this talk we discuss ConstraintSolver.jl, a new Julia package to tackle the problem of solving constraint programming problems purely in Julia.
This talk will present scaling and performance of the Oceananigans.jl ocean model on CPU and GPU systems. Oceananigans.jl is an all Julia code that is designed to study geophysical fluids problems ranging from idealized turbulence to planetary scale circulation. It uses the KernelAbstractions.jl package to support CPU and GPU single address space parallelism. It uses MPI.jl, to support multi-node and multi-GPU parallelism. MPI.jl is used both directly and through PencilArrays.jl.
With the recent advancements in power distribution, e.g., higher penetration of distributed energy resources (DERs), there is a significant demand for optimization tools to solve a variety of complex operational and planning problems, such as optimal dispatch, load shedding, and on-load tap changing. We have developed an optimization-focused approach to phase unbalanced power distribution modeling called PowerModelsDistribution, the design and usage of which we will introduce in this talk.
One of the main problems in AutoML implementation is finding the best strategy to search the most optimal pipeline in prediction or classification tasks. This problem is commonly known as CASH (Combined Algorithm Selection and Hyperparameter Optimization). This talk will show competitive results with significantly shorter computation time by just focusing the search in the model selection and structure of the pipeline without the need of hyperparameter optimization.
I will present an interactive web app for exploring phenotypes in flow cytometry data. In particular a multi-tissue, high-dimensional, immune cell dataset. This tool bridges computational methods in GigaSOM.jl and the popular FlowJo, used to annotate cells with gating strategies. By leveraging the geospatial mapping library OpenLayers to render, annotate and analyze cells, immunologists can now efficiently navigate the phenotype space of Human Cell Atlas datasets.
Designing vaccines is an expensive and time consuming process. This talk demonstrates how we can exploit automatic differentiation of ODEs, parallelization, stochastic search and Bayesian optimization to minimize post-vaccination invasive pneumococcal disease and antibiotic resistant strains in a bacteria population using a novel computational model of the bacterial population dynamics that integrates epidemiological and genomic data.
This talk is about a registered Julia package: PowerModelsDistributionStateEstimation.jl, that allows easy benchmark and design of state estimation models for power distribution systems. The goal is to accelerate the use of this technique in research and real-life settings. State estimation is formulated as a mathematical optimization problem using JuMP.jl and can be solved with off-the-shelf solvers. Different modeling options are featured, and the package is designed to be easily extensible.
We will show how Julia allows us to implement spatial branch-and-bound-type methods using interval arithmetic in parallel on GPUs, in a relatively painless way. As a test case, we calculate and verify existence and uniqueness of over one million stationary points of the transcendental Griewank function of two variables in one second on a recent GPU. We are not aware of any other system that is able to do this.
We present our code (LatticeQCD.jl) for quantum chromo-dynamics (QCD), which describes microscopic world inside of nucleons. QCD calculation has been implemented by Fortran and C++ on supercomputers or GPU clusters because it requires huge numerical resource, i.g. Monte-Carlo with inversions of matrices with 10^16 x 10^16, and has been succeeded to calculate crucial numbers used in experiments. We implemented a code for QCD in Julia, which achieves compatible speed with a Fortran code.
The ZX-calculus is a graphical language for representing and reasoning about quantum information. ZXCalculus.jl is a high-performance package for creating, manipulating, and visualizing ZX-diagrams in Julia. Comparing with a previous Python implementation PyZX, ZXCalculus.jl has 6-50x speed-ups on various tasks of simplifying ZX-diagrams. Moreover, this package is integrated with YaoCompiler.jl and works as a circuit simplification pass in the quantum compiler.
Determining one’s risk of developing various diseases throughout one’s lifetime is important for pursuing good health. An emerging method for performing this calculation is the Polygenic Risk Score, or PRS. A PRS method allows one to construct a model of risk of acquiring a certain disease given one’s own genome and provides a simple numerical result representing that risk. We will describe how we ported a widely used PRS program to Julia and the performance and usability that we gained.
Evolutionary relationships among organisms are depicted by a binary tree. However, not all species follow the paradigm of vertical inheritance of genes and thus, estimation of phylogenetic networks becomes necessary. PhyloNetworks is the first Julia package for the inference, manipulation, visualization, and use of phylogenetic networks. The package documentation has a full tutorial including upstream analyses, network estimation, bootstrap analysis, and downstream analyses for trait evolution.
ConstraintProgrammingExtensions.jl is a project bringing constraint programming to JuMP. Its main part is a large series of constraints that aim at providing a common interface for constraint-programming solvers. It also consists of a series of bridges that define relationships between those sets (including between high-level constraints such as knapsacks and mathematical-programming formulations) and of a FlatZinc reader-writer to import and export models in that common format.
We introduce ExaTron.jl which is a scalable GPU-MPI-based batch solver for many small nonlinear programming problems. We present ExaTron.jl's architecture, its kernel design principles, and implementation details with experimental results comparing different design choices. We demonstrate a linear scaling of parallel computational performance of ExaTron.jl on Summit at Oak Ridge National Laboratory.
Analog circuit simulation is widely used to design and verify analog circuits before they are manufactured. We present a novel, composable SPICE simulator written entirely in Julia, called JuliaSPICE.
The aim of this work is to evaluate the feasibility of re-implementing some key parts of the widely used Weather Research and Forecasting WRF-SFIRE simulator by replacing its core differential equations numerical solvers with state-of-the-art physics-informed machine learning techniques to solve ODEs and PDEs implemented in Julia, in order to transform it into a real-time simulator for wildfire spread prediction.
This talk introduces Fairness.jl, a toolkit to audit and mitigate bias in ML decision support tools. We shall introduce the problem of fairness in ML systems, its sources, significance and challenges. Then we will demonstrate Fairness.jl structure and workflow.
RandomBattles.jl is a Julia package for the efficient simulation of individual and team Player-vs-Player Battles in Pokemon Go, the AR mobile game by Niantic. This package can compute Monte Carlo simulations, as well as game theoretic solutions to perfect information games. Using the game’s structure and Nash equilibria, the algorithm computes optimal play strategies for an arbitrary number of moves. These simulations derived strategies that are highly similar to those employed by human players.
The talk presents the first data analysis in the LHCb experiment performed in Julia (arXiv:2107.03419). The analysis includes data selection, building and combining complex PDFs (
AlgebraPDF.jl), likelihood fitting, angular analysis (
PartialWaveFunctions.jl), hypotheses testing, and running pseudo experiments. New packages enrich Julia ecosystem fostering the adoption of the language in the High Energy Physics community.
Aquaculture, or the farmed production of fish and shellfish, has grown rapidly, from supplying just 7% of fish for human consumption in 1974 to more than half in 2016. Sustaining this rapid expansion requires data-driven management of the production process and environmental impacts. This talk presents a machine-learning-based exploration of environmental and fish behaviour datasets collected at three salmon farms in Norway, Scotland, and Canada using AutoML tools in Julia.
Interplay of linear algebra, machine learning, and HPC
The JuliaHPC community as a group maintains the infrastructure for using Julia in high-performance computing. In this BoF we invite newcomers, application developers, and maintainers to join us for an informal discussion around the state of Julia in HPC.
We present InvertibleNetworks.jl, an open-source package for invertible neural networks and normalizing flows using memory-efficient backpropagation. InvertibleNetworks.jl uses manually implement gradients to take advantage of the invertibility of building blocks, which allows for scaling to large-scale problem sizes. We present the architecture and features of the library and demonstrate its application to a variety of problems ranging from loop unrolling to uncertainty quantification.
This session will include all the experiences talks. List of all experiences talks: https://juliacon.org/2021/experiences/
So far, most nonlinear optimization modelers and solvers have primarily targeted CPU architectures. However, with the emergence of heterogeneous computing architectures, leveraging massively parallel accelerators in nonlinear optimization has become crucial for performance. As part of the Exascale Computing Project ExaSGD, we are studying how to efficiently run nonlinear optimization algorithms at exascale using GPU accelerators.
Registering a new release for your package is always a great moment. However, there are several challenges related with release management. In this talk, using the experience from JuliaData ecosystem, I will discuss the major things to consider if you want to keep your users happy.
This talk briefly presents OrbitalTrajectories.jl, a library providing tools for the analysis of orbital trajectories for space mission design. Making use of the Julia scientific modeling ecosystem to easily define and extend high-fidelity simulations of spacecraft motion, we demonstrate how key techniques including meta-programming, symbolic computation, non-linear optimisation, and automatic differentiation work towards generating, analysing, and stabilising orbital trajectories.
AtomicSets.jl, a Julian framework for structured convex optimization. Algorithms for structured optimization build up a solution from a set of prescribed atoms that represent simple structures. The atoms that participate in the final solution often represent key explanatory components of a model. We use Julia's dispatch system to implement a calculus of convex sets and their functional representations that compiles to efficient machine code.
For power grid operators, wind farms and solar farms are black boxes since details of inverter technologies, the main component of renewables, are proprietary info. Then, how can a grid operator assess a power grid's stability? We aim to design a toolbox to address this challenge. Julia Admittance will process experiment data and come up with admittance models for wind, solar, etc. With admittance models available, eigenvalues of the system will be computed to assess system stability.
JuliaFolds ecosystem supports structured parallelism for Julia with the packages such as Transducers.jl, Folds.jl, and FLoops.jl. It aims at providing parallelism that is easy-to-use, composable,and extensible. Furthermore, it provides a unified interface to different execution mechanisms such as multi-threading, GPU, and distributed parallelisms. In this talk, I discuss the composable design principle of the JuliaFolds packages.
We present a native-Julia nonlinear programming (NLP) solver MadNLP.jl. This solver implements the filter line-search interior-point method for constrained NLPs; to the best of our knowledge, MadNLP is currently the only native-Julia solver that is capable of handling general nonlinear equality/inequality-constrained optimization problems. MadNLP is interfaced with the algebraic modeling language JuMP.jl, the graph-based modeling language Plasmo.jl, and the NLP data structure NLPModels.jl.
Acsets are a novel infrastructure for handling data of different shapes, based on category theory and implemented in Catlab.jl. Acsets generalize both graphs and dataframes, and allow a much more general approach to data manipulation than was previously available. We will discuss both the mathematics of acsets and some of the metaprogramming techniques we used to implement them in Julia. Finally, we will give examples of how acsets have been key in developing many projects in AlgebraicJulia.
Soss is a probabilistic programming language (PPL) with first-class composable models. Through dynamic code generation, Soss can achieve speedup of several orders of magnitude in some models, for example using symbolic simplification of the log-density.
In this talk, we'll discuss the goals and design choices in Soss that distinguish it from other PPLs, followed by an overview of upcoming work.
Nonconvex.jl is a package that aims to interface all the major nonlinear and mixed integer nonlinear programming packages in Julia using a function-based API. Zygote.jl is used for automatic differentiation (AD) and ChainRules.jl can be used to define analytic gradients or custom adjoint rules for functions. Ipopt.jl, NLopt.jl, Percival.jl and Juniper.jl are some of the packages wrapped in Nonconvex.jl as of the writing of this abstract.
Are you interested in how machine learning can be used to predict the behavior of the "unpredictable" chaotic systems? This talk will be a deep dive into ReservoirComputing.jl (https://github.com/SciML/ReservoirComputing.jl), a package in the SciML ecosystem focused on a class of stabilized machine learning specialized for handling learning these difficult dynamical systems.
This is a BoF to talk about the various GPU programming packages in Julia:
If you have any thoughts or questions about these packages, or other approaches to GPU programming in Julia, please join this BoF to chat about it!
Using the earth’s magnetic field for navigation of aircraft has shown promise as a viable alternative to GPS. An airborne magnetic navigation system collects magnetic field data and uses predetermined magnetic maps to estimate location. A challenge arises when the measured data contains magnetic signals from both the (desired) earth field and (undesired) aircraft field. This work explores several approaches for obtaining a clean magnetic signal that is usable for navigation.
The NOMAD software is a derivative-free solver which implements the mesh adaptive direct search algorithm. Its purpose is to solve constrained problems where the objective and the functions defining the constraints correspond to the outputs of a program seen as a blackbox. This talk aims at presenting the NOMAD.jl interface to Julia, linked to the JuMP modeling language. Some applications will also be exposed.
Many optimization problems involve minimizing a sum of univariate functions, each with a different variable, subject to coupling constraints. We present PiecewiseQuadratics.jl and SeparableOptimization.jl, two Julia packages for solving such problems when these univariate functions in the objective are piecewise-quadratic.
How we currently implement the task parallel API in Julia introduces a couple of obstacles for supporting high-performance parallel programs. In particular, the compiler cannot analyze and optimize the child tasks in the context of the surrounding code. In this talk, I discuss our work on using Tapir (Schardl et al., 2019) to add parallelism to Julia that can be optimized by the compiler.
PrettyPrinting is a library for formatting composite data structures. PrettyPrinting optimizes the layout of the data to make it fit the screen width.
Out of the box, PrettyPrinting can format Julia code and standard Julia containers. It can be easily extended to format custom data types.
Scientific Machine Learning (SciML) is the branch of scientific computing that combines domain-aware and interpretable models with powerful machine learning techniques. The Julia language has been a key enabler of this burgeoning field, thanks to its unique SciML ecosystem. In this talk, we will present a contribution in this direction: an easy and flexible implementation of generative latent differential equations models.
Interpretable Compositional Networks (ICN), a variant of neural networks, that allows the user to get interpretable results, unlike regular artificial neural networks. An ICN is a glass-box producing functions composition that scale with the size of the input, allowing a learning phase on relatively small spaces. This presentation covers the different Julia packages and paradigms involved, a set of use-case, current limitations, future developments, and hopefully possible collaborations.
Join us on Gather.town for a social hour.
It is a virtual location where we will facilitate the poster sessions, social gatherings, and hackathon. You can join the space using the URL: https://gather.town/invite?token=3QYkt8gX.
You should have received the password through Eventbrite
While Monads make it easy to hide one context nicely in your code, with Extensible Effects you can combine multiple contexts and let them seamlessly interact with each other. TLDR: If you want to abstract and hide away some computational context, prefer Extensible Effects to Monads.
Macroeconomic modeling during the COVID-19 pandemic, and the switch to a new monetary policy framework, has required rapid adjustments to the DSGE.jl package, made possible by Julia’s flexible typing and efficient matrix computations. We review the new features in DSGE.jl that allow users to model periods of large economic shifts and uncertainty. As an illustration, we also explain how the Federal Reserve Bank of New York solved and estimated a model with these features during the recession.
Javis.jl is a general purpose animation library which builds on top of the Luxor.jl graphics library. It fills a gap in the Julia ecosystem by providing functionality to create object based animations to communicate complex ideas through simple means. Furthermore, Javis provides the flexibility for users to extend Javis’s visualizations to a variety of applications. Users are already expressing complicated ideas through winsome domain specific visuals such as planetary motion or brain mapping.
Using Julia code within private organizations can encounter challenges not faced in the open-source community. In this BoF, we'll be discussing the unique aspects of using Julia in private organizations and cover topics such as: production deployments, tooling/ techniques for teams coding in Julia, and answering questions regarding transition/adopting Julia within an organization.
Package latency remains one of the chief complaints among Julia users. While recent improvements in Julia have reduced the problem, the opportunity for additional progress is large. In this talk I'll analyze latency from a package developer's standpoint, describing some of the factors that affect latency and how improvements in package design can reduce it. I will briefly exhibit tools that can help identify opportunities for improvement.
NExOS.jl is a Julia package that implements the Nonconvex Exterior-point Operator Splitting (NExOS) algorithm (https://arxiv.org/pdf/2011.04552.pdf). The package is tailored for minimizing a convex cost function over a nonconvex constraint set, where projection onto the constraint set is single-valued around local minima.
We will present recent work in progress on guaranteed methods for inequality-constrained global nonlinear optimization in Julia. Using methods based on interval arithmetic allows us to guarantee (prove) that we return the true global minimum and minimizers for inequality-constrained optimization problems in low dimensions.
PackageCompiler.jl has become the de facto method for creating standalone Julia applications. In this talk, we will demonstrate the use of
PackageCompiler.jl to produce shared library bundles. This functionality was added recently and allows the easy creation of location-independent dynamic libraries which can be linked to and called from C, C++, Rust, or other languages which can link to and use C libraries.
Applications written in Julia targeted at a nonprofessional audience are still uncommon, even though libraries for designing such applications have existed for years. For software to be easy to use by laypeople, a simple installation process and an intuitive GUI are essential. We have been developing and deploying such an application for over three years. This talk will focus on our experiences during that time, how the situation has improved since Julia 0.6, and what it looks like today.
High frequency data typically exhibit asynchronous trading and microstructure noise, which can bias the covariances estimated by standard estimators. While a number of specialised estimators have been developed, they have had limited availability in open source software. HighFrequencyCovariance is the first Julia package which implements specialised estimators for volatility, correlation and covariance using high frequency financial data.
BLAS & LAPACK are an integral component of many numerical algorithms. Due to their importance, a lot of effort has gone into optimizing ASM/C/Fortran implementations. Nonetheless, early work demonstrated Julia implementations were often faster than competitors, while laying groundwork for new routines specialized for new problems. We discuss a roadmap toward providing Julia BLAS and LAPACK libraries, from optimizations in LoopVectorization to libraries like Octavian and RecursiveFactorization.
PGFPlots is a plotting package for LaTeX that produces plots with vector graphics and interfaces with the math typesetting of LaTeX.
PGFPlotsX.jl is a Julia plotting package that provides an interface to PGFPlots by transpiling Julia objects to LaTeX code. Furthermore, the figures generated by
PGFPlotsX.jl are directly rendered in IPython notebooks, Pluto and VSCode which allows for rapid plot prototyping. It also serves as one of the backends to the popular
The Julia community has embraced semantic versioning from a very early stage (if not from the get-go). The Julia package release flow has also been put through its paces and is easy to get started with. This is a great foundation and may be sufficient for most. However, some may require a more structured approach to release preparation, incorporating it into their day-to-day operations. This talk will show how to use the 'semantic release' framework for Julia packages to accomplish this.
In this talk, I illustrate a Julia workflow to study economic inequality and taxation in the United States. My workflow centers around Taxsim.jl which allows to answer a large number of research questions related to the US tax system. First, I import a widely used survey dataset to show how high and low before-tax incomes evolved since 1960. Next, using Taxsim.jl, I impute taxes paid to compare the evolution of after-tax incomes and to measure the redistributive effect of the tax system.
It takes the entire community to promote diversity and inclusion. This talk will focus on the current plans underway to promote diversity and inclusion in the Julia Community as well as give an updated look at the state of diversity and inclusion in our community.
Javis.jl is a graphical animation/visualization package for the Julia Language which is inspired by Grant Sanderson's (aka, famous Math educator and YouTuber, 3blue1brown) Python based animation engine, Manim. This talk is about the work I have been doing this summer to make Javis more friendly and feature rich for creators
Julia has historically been built against a single backing BLAS/LAPACK library, and switching to a different library has required a recompilation of Julia. This was compounded by issues with loading 3rd party binaries that linked against incompatible BLAS backends. This talk will showcase a new low-overhead compatibility layer in Julia v1.7 named libblastrampoline that allows for runtime switching of BLAS/LAPACK libraries, as well as allowing loading of multiple BLAS/LAPACK ABIs at once.
Makie.jl is a plotting package for high-performance interactive and publication-quality static data visualizations. MakieLayout, a former extension delivering flexible layouts and interactive widgets, has recently been integrated into the base package, and is now part of the default workflow. This talk will take a detailed look at the new syntax and the architecture behind the layout system, as well as highlight features that make creating complex multi-plot figures a breeze.
The Julia community aims to be welcoming, diverse, inclusive towards people from all backgrounds. However, the 2020 Julia User & Developer Survey found that only 3% of respondents were women, and reported no respondents who were non-binary or another gender. We, Julia Gender Inclusive, believe this needs to change. In this talk, we will share our ideas and initiatives for improving gender diversity among Julia users and developers, including outreach, community building, and mutual support.
The ability to create shared library bundles was recently added to
PackageCompiler.jl. In this talk, we will discuss the technical details of the implementation and give in-depth examples of using the resulting shared library bundles from C and Rust.
Graphs are a ubiquitous and versatile data structure, which allow representation of problems and systems across a vast array of domains, from infrastructure networks and molecules to language and social interactions. SuiteSparseGraphBLAS.jl casts graph computations as generalized linear algebra on sparse matrices. Support for ChainRules AD frameworks, and the wider ecosystem is a core feature of v1.0, releasing around JuliaCon.
The definition of the arithmetic operations defined in Julia assume that the arguments are not modified. However, in many situations, a variable represents an accumulator that can be modified to contain the result, e.g., when summing the elements of an array. Moreover, many types can be mutated and mutating the element may have significant performance benefit. This talk presents an interface that allows algorithms to exploit a possible mutability while still being completely generic.
JOSS, the Journal of Open Source Software (https://joss.theoj.org/) is a venue for publishing research software packages. This provides a mechanism for the large time investment required to develop open-source research software to be included within traditional systems for academic credit.
A JOSS paper is meant to be a short description of the contribution provided by the research software, with the main content being in the (archived) software repository itself.
Have you ever had a list of
Methods, e.g. from the output of
methodswith, and thought ”I just want to implement all of these, it would be great to use metaprogramming for that”?
ExprTools.jl has the parts to let you extract the info out of the method table, manipulate it, and then generate the AST you want for the new method you want to define.
Does this access undocumented Julia internals? Absolutely! Is this well tested? Comprehensively! Is this a good idea? Who knows!
Join us on Gather.town for a social hour.
It is a virtual location where we will facilitate the poster sessions, social gatherings, and hackathon. You can join the space using the URL: https://gather.town/invite?token=3QYkt8gX.
You should have received the password through Eventbrite