This is a decision in progress. Scroll down for a draft ATProto lexicon that represents releases.
We'd like to make it easier for people to try IO by publishing binaries that don't require Docker. Homebrew is one way that we can do this, but we need to support more platforms (especially Linux) and it seems too soon to start supporting multiple package managers. Instead, we prefer to build and release binaries for a few target architectures, initially darwin/arm64, linux/arm64, and linux/amd64.
For comparision, Hashicorp has a large download site at releases.hashicorp.com. It includes released binaries for multiple architectures, SHA hashes of these binaries, and signatures of the hashes. It seems useful to build something similar, but not necessary identical, to this.
Files can be stored in an existing cloud storage provider. We currently use Google Cloud Storage and have looked at DigitalOcean Spaces. These and other providers that we've considered don't automatically provide index pages, so we would need to generate those.
Instead of hosting our index pages on a separate domain, it seems best to generate them as part of the Agent IO website. We might put them under agent.io/releases (currently nonworking).
We need source material for these index pages, and rather than keep that data in private storage, we prefer to put it in a lexicon and store it in the @agent.io repo. It's all public information, anyway! This also eliminates the need to sign asset hashes, since everything in our ATProto repo is automatically signed by the repo signing key.
Pros
- We can start with a custom tool for this and later hook it up to CI.
- Release binaries are displayable on the Agent IO website.
- Asset publishing is decoupled from website generation, making both steps easier.
- Release pages could have well-defined URIs that could define standard.site documents.
- Other projects might find this lexicon useful.
- We could use this to distribute builds of other noteworthy binaries, such as Envoy.
Cons
- It's a little more work to maintain this vs. just using GitHub actions and a tool like
goreleaser, but that has other problems that make it difficult for us to use. - Asset publishing is decoupled from website generation, so we need to run both for releases to be visible.
Work In Progress
Here's our draft lexicon. It includes a list of binaries (assets) with URLs and hashes, freeform release notes, a structured list of changes that could be used to generate a change log, and a list of build dependencies.
{
"lexicon": 1,
"id": "io.agent.release",
"defs": {
"main": {
"type": "record",
"description": "A record representing a software release.",
"key": "tid",
"record": {
"type": "object",
"required": [
"name",
"version",
"createdAt"
],
"properties": {
"name": {
"description": "The name or identifier of the software.",
"type": "string",
"maxLength": 100
},
"version": {
"description": "The semantic version string, e.g., v1.2.0-beta.1",
"type": "string",
"maxLength": 50
},
"createdAt": {
"description": "Date and time the release was published.",
"type": "string",
"format": "datetime"
},
"notes": {
"description": "Release notes.",
"type": "string",
"format": "markdown",
"maxLength": 3000
},
"changes": {
"description": "Changelog entries for this release.",
"type": "ref",
"ref": "#changes"
},
"assets": {
"description": "List of download links associated with this release.",
"type": "array",
"items": {
"type": "ref",
"ref": "#asset"
}
},
"dependencies": {
"description": "The software dependencies of this release.",
"type": "array",
"items": {
"type": "ref",
"ref": "#dependency"
}
}
}
}
},
"asset": {
"description": "A link to a downloadable asset associated with a release.",
"type": "object",
"required": [
"name",
"url"
],
"properties": {
"name": {
"description": "Name of the asset (e.g., macos-x64, source-code).",
"type": "string"
},
"url": {
"description": "Direct download link.",
"type": "string",
"format": "uri"
},
"sha256": {
"description": "SHA-256 checksum.",
"type": "string"
}
}
},
"changes": {
"description": "Changelog entries for a release, informed by https://keepachangelog.com/en/1.1.0/.",
"type": "object",
"properties": {
"added": {
"description": "New features.",
"type": "array",
"items": {
"type": "string",
"format": "markdown",
"maxLength": 1000
}
},
"modified": {
"description": "Modifications of existing features.",
"type": "array",
"items": {
"type": "string",
"format": "markdown",
"maxLength": 1000
}
},
"deprecated": {
"description": "Deprecations.",
"type": "array",
"items": {
"type": "string",
"format": "markdown",
"maxLength": 1000
}
},
"removed": {
"description": "Removed features.",
"type": "array",
"items": {
"type": "string",
"format": "markdown",
"maxLength": 1000
}
},
"fixed": {
"description": "Fixed bugs.",
"type": "array",
"items": {
"type": "string",
"format": "markdown",
"maxLength": 1000
}
},
"security": {
"description": "Security updates.",
"type": "array",
"items": {
"type": "string",
"format": "markdown",
"maxLength": 1000
}
}
}
},
"dependency": {
"description": "A software dependency of the release",
"type": "object",
"required": [
"name",
"version"
],
"properties": {
"name": {
"description": "The name of the dependency",
"type": "string"
},
"version": {
"description": "The version of the dependency",
"type": "string"
}
}
}
}
}Source for this is in the agentio/lexicons repo.
Discussion
The build dependencies in our draft lexicon points to a larger topic of Software Bill of Materials, for which there are some notable existing formats:
Rather than replicate these larger formats, we suggest storing these files outside of the release record as additional assets that can be downloaded and verified with a hash.