2 Distributing Racket Variants
This chapter is about distributing variants of Racket, as opposed to distributing applications that are built with Racket. See raco distribute: Sharing Stand-Alone Executables for information about distributing applications.
Important: To build installers that can be distributed to other users, do not use make in-place or make unix-style, but instead start from a clean repository.
Use one non-Windows machine as a server, where packages will be pre-built. Then, as described below, create platform-specific installers on some number of client machines, each of which contacts the server machine to obtain pre-built packages. The server can act as a client, naturally, to create an installer for the server’s platform.
The distribution-build process is a collaboration between the Racket Git repository’s top-level makefile and the "distro-build" package.
2.1 Running Build Farms
The installers target of the makefile will do everything to generate installers: build a server on the current machine, run clients on hosts specified via CONFIG, and start/stop VirtualBox virtual machines or Docker containers that act as client machines.
If the server is already built, the installers-from-built target will drive the client builds without re-building the server.
See the documentation of the "distro-build" package for a description of the site-configuration module and requirements on client hosts.
If "my-site-config.rkt" is a configuration module, then
make installers CONFIG=my-site-config.rkt
drives the build farm, and the resulting installers are in "build/installers", with a hash table mapping descriptions to installer filenames in "build/installer/table.rktd". A log file for each client is written to "build/log".
If you have the "distro-build-server" package installed in some Racket build (not the one for building installers), you can use
make describe-clients CONFIG=my-site-config.rkt
to see, without building anything, the effect of the configuration in "my-site-config.rkt" and the planned build steps. See also the #:fake-installers? site-configuration option.
The default CONFIG path is "build/site.rkt", so you could put your configuration file there and omit the CONFIG argument to make. A default configuration file is created there automatically. Supply CONFIG_MODE=... to pass a configuration mode on to your site-configuration module (accessible via the current-mode parameter). Supply CLEAN_MODE=--clean to make the default #:clean? configuration for a client to #t instead of #f, supply RELEASE_MODE=--release to make the default #:release? configuration #t, supply SOURCE_MODE=--source to make the default #:source? configuration #t, and supply VERSIONLESS_MODE=--version to make the default #:versionless? configuration #t.
A configuration file can specify the packages to include, host address of the server, distribution name, installer directory, and documentation search URL, but defaults can be provided as make arguments via PKGS, SERVER plus SERVER_PORT plus SERVER_HOSTS, DIST_NAME, DIST_BASE, and DIST_DIR, DOC_SEARCH, respectively. The site configuration’s top-level options for packages and documentation search URL are used to configure the set of packages that are available to client machines to include in installers.
For each installer written to "build/installers", the installer’s name is
"‹dist-base›-‹version›-‹platform›-‹dist-suffix›.‹ext›"
where ‹dist-base› defaults to "racket" (but can be set via DIST_BASE), ‹platform› is from (system-library-subpath #f) but normalizing the Windows results to "i386-win32" and "x86_63-win32", -‹dist-suffix› is omitted unless a #:dist-suffix string is specified for the client in the site configuration, and ‹ext› is platform-specific: ".sh" for Unix (including Linux), ".dmg" or ".pkg" for Mac OS, and ".exe" for Windows.
The server supports both 'cs and '3m clients by creating built packages in machine-independent form (which is then recompiled to the client’s native format, still much faster than compiling from source). Set SERVER_COMPILE_MACHINE= to disable machine-independent format for built packages.
2.2 Generating Installer Web Sites
The site target of the makefile uses the installers target to generate a set of installers, and then it combines the installers, packages, a package catalog, and log files into a directory that is suitable for access via a web server.
Supply the same CONFIG=... and CONFIG_MODE=... arguments for site as for installers. The configuration file should have a #:dist-base-url entry for the URL where installers and packages will be made available; the installers target uses #:dist-base-url to embed suitable configuration into the installers. Specifically, installers are configured to access pre-built packages and documentation from the site indicated by #:dist-base-url.
Note that #:dist-base-url should almost always end with "/", since other URLs will be constructed as relative to #:dist-base-url.
The site is generated as "build/site" by default. A #:site-dest entry in the configuration file can select an alternate destination.
Use the site-from-installers makefile target to perform the part of site that happens after installers (i.e., to generate a site from an already-generated set of installers).
2.3 Managing Snapshot Web Sites
The snapshot-site makefile target uses site (so supply the same CONFIG=... and CONFIG_MODE=... arguments), and then treats the resulting site as a snapshot with additional snapshot-management tasks.
For snapshot management, the destination of the files generated for site (as specified by #:site-dest) should be within a directory of snapshots. The configuration file can use (current-stamp) to get a string that represents the current build, and then use the string both for #:dist-base-url and #:site-dest. Normally, the stamp string is a combination of the date and Git commit hash.
Snapshot management includes creating an "index.html" file in the snapshots directory (essentially a copy of the snapshot’s own "index.html") and pruning snapshot subdirectories to keep the number of snapshots at the amount specified by #:max-snapshots configuration-file entry (with a default value of 5).
Use the snapshot-at-site makefile target to perform the part of snapshot-site that happens after site (i.e., to manage snapshots around an already-generated site).
2.4 Separate Server and Clients
Instead of using the installers makefile target and a site configuration file, you can run server and client processes manually.
Roughly, the steps are as follows
On the server machine:
make server PKGS="‹pkgs›"
See step 2 in the detailed steps below for more information on variables other than PKGS that you can provide with make.
On each client machine:
make client SERVER=‹address› PKGS="‹pkgs›"
See 4 in the detailed steps below for more information on variables other than SERVER and PKGS that you can provide with make.
In more detail, the steps are as follows:
Build racket on a server.
The base target of the makefile will do that, if you haven’t done it already. (The server only works on non-Windows platforms, currently.)
On the server, build packages and start a catalog server.
The server-from-base target of the makefile will do that.
Alternatively, use the server target, which combines base and server-from-base (i.e., steps 1 and 2).
The SERVER_PORT variable of the makefile choose the port on which the server listens to clients. The default is port 9440.
The SERVER_HOSTS variable of the makefile determines the interfaces at which the server listens. The default is localhost which listens only on the loopback device (for security). Supply the empty string to listen on all interfaces. Supply multiple addresses by separating them with a comma.
The PKGS variable of the makefile determines which packages are built for potential inclusion in a distribution.
The DOC_SEARCH variable of the makefile determine a URL that is embedded in rendered documentation for cases where a remote search is needed (because other documentation is not installed).
The SRC_CATALOG variable determines the catalog that is used to get package sources and native-library packages. The default is http://pkgs.racket-lang.org.
The SERVER_PKG_INSTALL_OPTIONS variable determines extra flags that are passed to raco pkg install when installing on the server (to create package builds that are sent to clients). For example, SERVER_PKG_INSTALL_OPTIONS=--source could be useful to ensure that the server always builds from sources.
The PACK_BUILT_OPTIONS variable can be set to --mode ‹mode› to set the package mode for built packages. The default infer mode infers uses the package’s distribution-preference "info.rkt" field, if any, infers binary if the package has any native libraries and no Racket sources, and infers built otherwise.
The server provides README files from the "build/readmes" directory. If "README.txt" does not exist when the sever is started, a default file is created (and clients download "README.txt" by default).
If you stop the server and want to restart it, use the built-package-server makefile target instead of starting over with the server target.
On each client (one for each platform to bundle), build racket.
This is the same as step 1, but on each client. If the client and server are the same, there’s nothing more to do for step 3.
On each client, create an installer.
The client target of the makefile will do that.
Provide SERVER as the hostname of the server machine, but a localhost-based tunnel back to the server is more secure and avoids the need to specify SERVER_HOSTS when starting the server in step 2. Also, provide SERVER_PORT if an alternate port was specified in step 2.
Provide the same PKGS (or a subset) as in step 2 if you want a different set than the ones listed in the makefile. Similarly, DOC_SEARCH normally should be the same as in step 2, but for a client, it affects future documentation builds in the installation.
Alternatively, use the client target, which combines base and client-from-base (i.e., steps 3 and 4).
On Windows, you need NSIS installed, either in the usual location or with makensis in your command-line path.
To create a release installer, provide RELEASE_MODE as --release to make. A release installer has slightly different defaults that are suitable for infrequently updated release installations, as opposed to frequently updated snapshot installations.
To create a source archive, provide SOURCE_MODE as --source to make.
To create an archive that omits the version number and also omit and version number in installer paths, provide VERSIONLESS_MODE as --versionless to make.
To change the human-readable name of the distribution as embedded in the installer, provide DIST_NAME to make. The default distribution name is Racket. Whatever name you pick, the Racket version number is automatically added for various contexts.
To change the base name of the installer file, provide DIST_BASE to make. The default is racket.
To change the directory name for installation on Unix (including Linux), provide DIST_DIR to make. The default is racket.
To add an extra piece to the installer’s name, such as an identifier for a variant of Linux, provide DIST_SUFFIX to make. The default is "", which omits the prefix and its preceding hyphen.
To set the description string for the installer, provide DIST_DESC to make. The description string is recorded alongside the installer.
To set the initial package catalogs URLs for an installation, provide DIST_CATALOGS_q to make. Separate multiple URLs with a space, and use an empty string in place of a URL to indicate that the default catalogs should be used. The _q in the variable name indicates that its value can include double quotes (but not single quotes)—
which are needed to specify an empty string, for example. To select a "README" file for the client, provide README to make. The README value is used as a file name to download from the server.
To create a ".tgz" archive instead of an installer (or any platform), set TGZ_MODE to --tgz.
For a Mac OS installer, set SIGN_IDENTITY as the name to which the signing certificate is associated. Set MAC_PKG_MODE to --mac-pkg to create a ".pkg" installer instead of a ".dmg" image.
For a Windows installer, set OSSLSIGNCODE_ARGS_BASE64 as a Base64 encoding of an S-expression for a list of argument strings for osslsigncode. The -n, -t, -in, and -out arguments are provided to osslsigncode automatically, so supply the others.
The SERVER_CATALOG_PATH and SERVER_COLLECTS_PATH makefile variables specify paths at SERVER plus SERVER_PORT to access the package catalog and pre-built "collects" tree needed for a client, but those paths should be empty for a server started with make server, and they are used mainly by make client-from-site (described below).
The UPLOAD makefile variable specifies a URL to use as an upload destination for the created installed, where the installer’s name is added to the end of the URL, or leave as empty for no upload.
On each client, step 4 produces a "bundle/installer.txt" file that contains the path to the generated installer on one line, followed by the description on a second line. The installer is also uploaded to the server, which leaves the installer in a "build/installers" directory and records a mapping from the installer’s description to its filename in "build/installers/table.rktd".
If you provide JOB_OPTIONS=‹options› for either a client or server build, the options are used both for raco setup and raco pkg install. Normally, JOB_OPTIONS is used to control parallelism.
2.5 Creating a Client from an Installer Web Site
If you (or someone else) previously created an installer site with make site, then make client-from-site in a clean repository creates an installer for the current platform drawing packages from the site.
At a minimum, provide SERVER, SERVER_PORT (usually 80 or 443), SERVER_URL_SCHEME (if https instead of http) and SITE_PATH (if not empty, include a trailing /) makefile variables to access a site at
http://$(SERVER):$(SERVER_PORT)/$(SITE_PATH)
The client-from-site makefile target chains to make client while passing suitable values for DIST_CATALOGS_q, DOC_SEARCH, SERVER_CATALOG_PATH, and SERVER_COLLECTS_PATH. Supply any other suitable variables, such as DIST_NAME or RELEASE_MODE, the same as for make client.