18 Sources🔗ℹ

A source is a value that implements gen:source. When used with fetch, a source produces an input port and an estimate of how many bytes that port can produce. Denxi uses sources to read data with safety limits. To tap a source means gaining a reference to the input port and estimate. To exhaust a source means gaining a reference to a contextual error value. We can also say a source is tapped or exhausted.

Note that these terms are linguistic conveniences. There is no value representing a tapped or exhausted state. The only difference is where control ends up in the program, and what references become available as a result of using fetch on a source.

The procedure is given an input port, and an estimate of the maximum number of bytes the port can produce. This estimate could be +inf.0 to allow unlimited reading, provided the user allows this in their configuration.

The sole argument to the procedure depends on the source type.

fetch attempts to tap source. If successful, fetch calls tap in tail position, passing the input port and the estimated maximum number of bytes that port is expected to produce. Otherwise, fetch calls exhaust in tail position using a source-dependent argument.

identify attempts to return an input port that produces bytes used to identify a source? value. The input port does not produce bytes for expected content, and cannot use any information outside of what’s available in the value itself. This allows other other systems to compute cache keys and avoid unnecessary calls to fetch. If identify returns #f for a source, then that source cannot be identified.

The computed value of the subprogram is FAILURE if the source is exhausted. Otherwise, the value is what’s returned from tap.

The log will gain a ($fetch id errors) message, where errors is empty if the fetch is successful.

This is a front-end to identify that will always produce a fixed-length byte string. Prefer using this over using identify directly.

The default value will infer if the string is suitable for use with file-source or http-source, in that order. If an error is raised, it will be returned within an exhausted-source.

18.1 Source and Fetch Settings🔗ℹ

18.2 Defining Source Types🔗ℹ

On expansion, define-source defines a new structure type using (struct id (field ...)). The type is created with a guard that enforces per-field contracts. Instances implement gen:source.

define-source injects several bindings into the lexical context of body:

• %src, %tap, and %fail are each bound to their respective formal argument of fetch.

• %fetch is (bind-recursive-fetch %tap %fail).

• Each field identifier is bound to a respective value for an instance of the structure type.

To understand how these injected bindings work together, let’s go through a few examples.

Use %tap to fulfil data with an input port and an estimated data length. In the simplest case, you can return constant data.

byte-source uses %tap like so:

(define-source #:key byte-source-data (byte-source [data bytes?])

(%tap (open-input-bytes data)

(bytes-length data)))

Notice that the data is used to both define a data field (where it appears by bytes?) and to reference the value contained in that field (within open-input-bytes and bytes-length).

Use %fail in tail position with error information to indicate a source was exhausted.

file-source uses %fail like so:

(define-source #:key file-source-path (file-source [path path-string?])

(with-handlers ([exn:fail:filesystem? %fail])

(%tap (open-input-file path)

(+ (* 20 1024)

(file-size path)))))

Note that %fail is an exhaust/c procedure, so it does not have to be given an exception as an argument.

%fetch is a recursive variant of fetch that uses %tap, but a possibly different exhaust/c procedure. This allows sources to control an entire fetch process and fall back to alternatives.

first-available-source uses a resursive fetch to iterate on available sources until it has none left to check.

(define-source #:key first-available-source-sources

(first-available-source [available (listof source?)] [errors list?])

(if (null? available)

(%fail (reverse errors))

(%fetch (car available)

(λ (e)

(%fetch (first-available-source (cdr available) (cons e errors))

%fail)))))

Finally, %src is just a reference to an instance of the structure containing each field.

compute-key is an expression that must resolve to a procedure, or #f. It is used to generate an implementation of identify, such that the procedure must produce a value that can be reasonably coerced to an input port for use with identify. Currently, the accepted values include input ports, paths, strings, byte strings, instances of url, sources, or lists of the aforementioned types.

lock-source works in the context of fetch, using the provided exhaust procedure.

18.3 Source Types🔗ℹ

If all sources for an instance are exhausted, then the instance is exhausted. As sources are visited, errors are functionally accumulated in errors.

The value produced for an exhausted first-available-source is the longest possible list bound to errors.

(define src

(lines-source "\r\n"

'("#lang racket/base"

"(provide a)"

"(define a 1)")))

; "#lang racket/base (provide a) (define a 1) "

(fetch src consume void)

If the source is exhausted, it yields a relevant exn:fail:filesystem exception.

If request-url has the "file" scheme, then http-source behaves like file-source. In this case, only the URL path is used from request-url.

If the source is exhausted, it yields a relevant exception.

The behavior of the source is impacted by DENXI_DOWNLOAD_MAX_REDIRECTS.

18.4 Source Expressions🔗ℹ

The following procedures are useful for declaring sources in a package input.

• If variant is a source, then the returned value is variant.

• If variant is a string, then the returned value is ((current-string->source) variant)

• If variant is a byte string, then the returned value is (byte-source variant).

• If variant is a path, then the returned value is (file-source variant).

Due to this behavior, from-file will return different results when the containing source file changes location on disk.

18.5 Untrusted Source Expressions🔗ℹ

If the evaluation produces a source, then the result of the subprogram is that source, and no other messages will appear in the subprogram log.

If the evaluation does not produce a source, then the result is FAILURE and the subprogram log gains a ($bad-source-eval 'invariant datum).

If the evaluation is blocked by the security guard, then the result is FAILURE and the subprogram log gains a ($bad-source-eval 'security datum).