Build Codex

Table of Contents

• Install developer tools
  • Linux
  • macOS
  • Windows + MSYS2
  • Other
• Clone and prepare the Git repository
• Build the executable
• Run the example

Optional

• Run the tests

Prerequisites

To build nim-codex, developer tools need to be installed and accessible in the OS.

Instructions below correspond roughly to environmental setups in nim-codex's CI workflow and are known to work.

Other approaches may be viable. On macOS, some users may prefer MacPorts to Homebrew. On Windows, rather than use MSYS2, some users may prefer to install developer tools with winget, Scoop, or Chocolatey, or download installers for e.g. Make and CMake while otherwise relying on official Windows developer tools. Community contributions to these docs and our build system are welcome!

Rust

The current implementation of Codex's zero-knowledge proving circuit requires the installation of rust v1.79.0 or greater. Be sure to install it for your OS and add it to your terminal's path such that the command cargo --version gives a compatible version.

Linux

WARNING

Linux builds currently require gcc $\le$ 13. If this is not an option in your system, you can try building within Docker as a workaround.

Package manager commands may require sudo depending on OS setup.

On a bare bones installation of Debian (or a distribution derived from Debian, such as Ubuntu), run

apt-get update && apt-get install build-essential cmake curl git rustc cargo

Non-Debian distributions have different package managers: apk, dnf, pacman, rpm, yum, etc.

For example, on a bare bones installation of Fedora, run

dnf install @development-tools cmake gcc-c++ rust cargo

In case your distribution does not provide required Rust version, we may install it using rustup

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs/ | sh -s -- --default-toolchain=1.79.0 -y

. "$HOME/.cargo/env"

Note that you will currently not be able to build Codex with gcc 14. To verify that you have a supported version, run:

gcc --version

If you get a number that starts with 14 (e.g. 14.2.0), then you need to either downgrade, or try a workaround like building within Docker.

macOS

Install the Xcode Command Line Tools by opening a terminal and running

xcode-select --install

Install Homebrew (brew) and in a new terminal run

brew install bash cmake rust

Check that PATH is setup correctly

which bash cmake

# /usr/local/bin/bash
# /usr/local/bin/cmake

Windows + MSYS2

Instructions below assume the OS is 64-bit Windows and that the hardware or VM is x86-64 compatible.

Download and run the installer from msys2.org.

Launch an MSYS2 environment. UCRT64 is generally recommended: from the Windows Start menu select MSYS2 MinGW UCRT x64.

Assuming a UCRT64 environment, in Bash run

pacman -Suy
pacman -S base-devel git unzip mingw-w64-ucrt-x86_64-toolchain mingw-w64-ucrt-x86_64-cmake mingw-w64-ucrt-x86_64-rust

We should downgrade GCC to version 13 [^gcc-14]

pacman -U --noconfirm \
  https://repo.msys2.org/mingw/ucrt64/mingw-w64-ucrt-x86_64-gcc-13.2.0-6-any.pkg.tar.zst \
  https://repo.msys2.org/mingw/ucrt64/mingw-w64-ucrt-x86_64-gcc-libs-13.2.0-6-any.pkg.tar.zst

Optional: VSCode Terminal integration

You can link the MSYS2-UCRT64 terminal into VSCode by modifying the configuration file as shown below. File: C:/Users/<username>/AppData/Roaming/Code/User/settings.json

{
    ...
    "terminal.integrated.profiles.windows": {
      ...
      "MSYS2-UCRT64": {
        "path": "C:\\msys64\\usr\\bin\\bash.exe",
        "args": [
          "--login",
          "-i"
        ],
        "env": {
          "MSYSTEM": "UCRT64",
          "CHERE_INVOKING": "1",
          "MSYS2_PATH_TYPE": "inherit"
        }
      }
    }
}

Other

It is possible that nim-codex can be built and run on other platforms supported by the Nim language: BSD family, older versions of Windows, etc. There has not been sufficient experimentation with nim-codex on such platforms, so instructions are not provided. Community contributions to these docs and our build system are welcome!

Repository

In Bash run

git clone https://github.com/codex-storage/nim-codex.git repos/nim-codex && cd repos/nim-codex

nim-codex uses the nimbus-build-system, so next run

make update

This step can take a while to complete because by default it builds the Nim compiler.

To see more output from make pass V=1. This works for all make targets in projects using the nimbus-build-system

make V=1 update

Executable

In Bash run

make

The default make target creates the build/codex executable.

Tools

Circuit download tool

To build the circuit download tool located in tools/cirdl run:

make cirdl

Example usage

See the instructions in the Quick Start.

Tests

In Bash run

make test

testAll

Prerequisites

To run the integration tests, an Ethereum test node is required. Follow these instructions to set it up.

Windows (do this before 'All platforms')

1. Download and install Visual Studio 2017 or newer. (Not VSCode!) In the Workloads overview, enable Desktop development with C++. ( https://visualstudio.microsoft.com )

All platforms

1. Install NodeJS (tested with v18.14.0), consider using NVM as a version manager. Node Version Manager (nvm)
2. Open a terminal
3. Go to the vendor/codex-contracts-eth folder: cd /<git-root>/vendor/codex-contracts-eth/
4. npm install -> Should complete with the number of packages added and an overview of known vulnerabilities.
5. npm test -> Should output test results. May take a minute.

Before the integration tests are started, you must start the Ethereum test node manually.

1. Open a terminal
2. Go to the vendor/codex-contracts-eth folder: cd /<git-root>/vendor/codex-contracts-eth/
3. npm start -> This should launch Hardhat, and output a number of keys and a warning message.

Run

The testAll target runs the same tests as make test and also runs tests for nim-codex's Ethereum contracts, as well a basic suite of integration tests.

To run make testAll.

Use a new terminal to run:

make testAll

Building Within Docker

For the specific case of Linux distributions which ship with gcc 14 and a downgrade to 13 is not possible/desirable, building within a Docker container and pulling the binaries out by copying or mounting remains an option; e.g.:

# Clone original repo.
git clone https://github.com/codex-storage/nim-codex

# Build inside docker
docker build -t codexstorage/nim-codex:latest -f nim-codex/docker/codex.Dockerfile nim-codex

# Extract executable
docker create --name=codex-build codexstorage/nim-codex:latest
docker cp codex-build:/usr/local/bin/codex ./codex
docker cp codex-build:/usr/local/bin/cirdl ./cirdl

and voilà, you should have the binaries available in the current folder.