Live data

loadTable and loadJSON read a document once and keep it. A DataFeed reads one address over and over, so a sketch can draw something that is true right now. That could be the tide, a day of earthquakes, or a number a machine down the hall is publishing. When the machine wants to send rather than answer, use a PushFeed instead. It holds a connection open, so each message arrives the moment it is sent.

The request runs on a background queue, so draw() never waits for the network. The bytes are read into a JSON or a Table on that queue too, so no frame pays for the parse.

Nothing throws. Until the first answer arrives, json reads as null and table, text, and bytes are nil, even when the network is down. A feed with nothing to draw is in one state, not two. A failure after that keeps the last answer. problem says why.

Contents

• DataFeed
• Reading the answer
• Telling news from a quiet poll
• How it is going
• What the bytes are read as
• How often it asks
• PushFeed: a feed that is pushed
• Messages, one by one
• Staying connected
• In an export
• Sending a key or a contact
• Sandboxed apps

DataFeed

DataFeed(_ address: String, every interval: Double = 300,
         as content: Content = .auto, headers: [String: String] = [:])
DataFeed(_ url: URL, every interval: Double = 300,
         as content: Content = .auto, headers: [String: String] = [:])

Make one in setup(), start() it, then read the latest answer in draw().

final class Tide: Sketch {
    private let tide = DataFeed("https://example.org/tide.json", every: 600)

    override func setup() {
        tide.start()
    }

    override func draw() {
        background(.white)
        let height = tide.json["height"].number ?? 0
        drawCircle(center: center, radius: 40 + height * 20)
    }
}

every: is in seconds and never goes below one. Text that is not a valid address leaves the feed empty, and problem says so, the same as a server that never answers.

Reading the answer

json and table are the same types that loadJSON and loadTable hand back. A sketch that started from a bundled file can read a feed with the code it already has. text is filled whenever the bytes are UTF-8, whatever else was parsed.

Telling news from a quiet poll

updateCount counts the answers that differed from the one before. A poll that brings back what the feed already had does not count, so a sketch can use the count to start an entrance:

if tide.updateCount != seen {
    seen = tide.updateCount
    startTheTransition()
}

timeSinceUpdate is seconds since the answer last changed, or nil before the first one.

How it is going

A failure never clears what the feed already holds. A sketch can keep drawing the last good answer with the notice over it, instead of going blank:

if let problem = tide.problem { drawStatus(problem, style: .warning) }

What the bytes are read as

as: decides. The default, .auto, takes the server's word for it, and falls back to the bytes.

Name the content type yourself when a server labels its documents carelessly, which many servers do.

How often it asks

Polling is slow on purpose, because a sketch on a wall runs for weeks.

• every: never goes below one second.
• The next request is scheduled only once the last one has finished, so a slow server can never make requests stack up.
• The feed asks conditionally. It sends back the ETag or Last-Modified the server gave it, so a server that supports those can reply with a header and no body. That reply is not an update.
• A run of failures backs off. The wait doubles each time, up to eight times the interval, and the next answer puts it back to the plain interval.

Keep a feed small. The parse happens off the frame, but a document of many megabytes is still slow to fetch and read. Put a big document in a file and read it once in setup().

PushFeed: a feed that is pushed

PushFeed(_ address: String, headers: [String: String] = [:],
         greeting: String? = nil, retryEvery: Double = 3)
PushFeed(_ url: URL, headers: [String: String] = [:],
         greeting: String? = nil, retryEvery: Double = 3)

A DataFeed asks on a schedule, which suits a value that changes slowly. A machine that wants to speak the moment something happens needs a connection held open instead, and that is what a PushFeed gives it. The address decides how the connection is made. ws:// and wss:// open a web socket. Anything else is read as a stream of server-sent events, which is the plain-HTTP way a server pushes.

final class Edits: Sketch {
    private let edits = PushFeed("https://stream.wikimedia.org/v2/stream/recentchange")

    override func setup() {
        edits.start()
    }

    override func draw() {
        background(.black)
        for message in edits.messages() {
            splash(message.json["title"].text ?? "")
        }
    }
}

The reads you learned on a DataFeed still work here. json, text, and bytes hold the latest message, problem says why when something is wrong, and nothing throws. event adds the label, on a stream that names its events.

Messages, one by one

More than one message can arrive between two frames, and the latest-message reads only show the last of them. messages() misses nothing. It hands back every message since the last call, oldest first, and each one is a Message carrying bytes, text, json, and event.

for message in edits.messages() {
    ripples.append(Ripple(title: message.json["title"].text ?? ""))
}

A feed that nobody drains keeps the newest few hundred messages and drops the oldest, so an undrained buffer never grows without bound.

updateCount counts every message, and on a PushFeed a repeat still counts. A poll can bring back what a feed already had, but a push is sent because the server had something to say. timeSinceUpdate is seconds since the last message, or nil before the first one.

Staying connected

A sketch on a wall outlives any socket, so a PushFeed has to reconnect. The feed does all of it on its own, and a sketch only ever reads isConnected and problem to say what is happening.

• A dropped connection redials after retryEvery: seconds, which the feed reads back as retryInterval and which never goes below one. A run of failures doubles the wait each time, up to eight times that. Anything that arrives puts the wait back to retryEvery: seconds.
• A stream of server-sent events that names its own retry time is obeyed. One that labels its messages with ids is resumed with the last id seen, so a short drop loses nothing the server still holds.
• A web socket is pinged every few seconds, so a connection that died without a word is noticed and redialed instead of being trusted. A quiet stream of events is given a minute before the same treatment.
• greeting: is sent each time the connection opens, not once. A service that wants a subscribe message wants it again after every redial. That is why the greeting is part of the feed, rather than a send(_:) made in setup().
• A server that ends the stream on purpose (a 204) is honored. The feed stops instead of redialing.

In an export

In a headless export the feed reads once, while start() runs, and holds that answer for every frame. timeSinceUpdate reads zero throughout. An export that fetched per frame would render differently every run, and the export tier is built so that it does not. The read blocks whichever thread called start(), which is the thread running setup() in the ordinary case.

Sending a key or a contact

headers: are sent with every request. That is where an API key goes, or the contact address that some public services ask for:

let birds = DataFeed("https://example.org/api/birds", every: 900,
                     headers: ["X-Api-Key": key])

Keep a real key out of the sketch file. Read it from the environment or a file you do not commit.

Sandboxed apps

A sketch you run from the terminal reaches the network freely. A sketch wrapped as a sandboxed app needs the outgoing-connections entitlement (com.apple.security.network.client) in its entitlements file. Without that entitlement every request fails, and problem says the connection was refused.

See also

• Data - loadTable and loadJSON, which read a document once and hand back the same types a feed does
• Parameters - @Param parameters, for values you tune rather than fetch
• OSC - values pushed at the sketch over the network, where a feed pulls them
• Installation - what else changes for a sketch that runs for weeks