The package mirror is filled by external-repos import-lightwell-packages, which reads Maven from Pulp and Python and npm from Tang. This doc covers the manual local seed and how that job is meant to run.
The seed is not applied by dbmigrate. scripts/create_lightwell_repo.sh creates the Pulp repositories, imports them into the app, loads vulnerabilities, advisories, and package rows from SQL (so Pulp can be seeded from those coordinates), uploads that catalog into Pulp, then runs import-lightwell-packages --force so the mirror matches Pulp:
./scripts/create_lightwell_repo.sh
The database seed uses the compose database on port 5433. Override it with LIGHTWELL_SEED_PSQL when your configs/config.yaml uses different settings.
Packages are written onto the repositories from pkg/external_repos/lightwell_repos.json, in org -3:
| Name | Packages |
|---|---|
lightwell/java/validated |
Catalog versions from validated_catalog in db/seeds/lightwell_packages.sql. Empty release. No advisories. |
lightwell/java/remediated |
Advisory fixed_versions, including rhlw. |
lightwell/java/predisclosure |
Advisory fixed_versions, including rhlw. |
lightwell/python/validated |
Catalog versions from validated_catalog in db/seeds/lightwell_packages.sql. Empty release. No advisories. |
lightwell/python/remediated |
Advisory fixed_versions, including rhlw. |
lightwell/python/predisclosure |
Advisory fixed_versions, including rhlw. |
lightwell/javascript/validated |
Catalog versions from validated_catalog in db/seeds/lightwell_packages.sql. Empty release. No advisories. |
lightwell/javascript/remediated |
JavaScript advisory fixed_versions, including rhlw. |
JavaScript uses the language name in repository names and Pulp paths (javascript/validated, javascript/remediated), and npm as the API content type and ecosystem filter. This follows Java’s java/validated path with content type maven. These repositories were previously named lightwell/npm/*; the rename tracks the Pulp base paths. Migration 20261110120000_delete_lightwell_npm_repos removes the rows left behind under the old names.
After deployment, run the existing repository import (make repos-import) to register the JavaScript distributions. Ensure options.lightwell_ignore_repos contains neither lightwell/javascript/validated nor lightwell/javascript/remediated. Entitled callers can then retrieve them through the Lightwell repositories endpoint with content_type=npm. The existing import-lightwell-packages job populates the package catalog; package detail routes use js_packages.
Each version row has its own summary, license, project URL, and author. Python and npm rows also set author_email. Maven author_email is empty. JavaScript advisories are attached to lightwell/javascript/remediated.
The SQL package seed defines which versions go into Pulp: Maven POMs, Python wheels under scripts/fixtures/python_wheels/, and npm tarballs generated by scripts/create_lightwell_repo.sh. import-lightwell-packages then replaces those seed rows with the mirror import. Repository package list, versions, and detail read that mirror. The import still reads Maven from Pulp and Python and npm from Tang.
scripts/seed_lightwell_dev.sh reloads only the database seed. It skips the vulnerability inserts when those rows already exist. The catalog repositories must already be present. After changing package coordinates, re-run create_lightwell_repo.sh (or re-seed Pulp and import-lightwell-packages --force) so Pulp and the mirror stay aligned.
One cron runs import-lightwell-packages every 15 minutes with concurrencyPolicy: Forbid. It imports Maven, Python, and npm. If a run no longer finishes inside that interval, split it into three jobs using --ecosystem maven, --ecosystem python, and --ecosystem npm. Two jobs must never import the same repository.
OPTIONS_LIGHTWELL_PACKAGE_IMPORT_FORCE_AT forces one import without changing the cron command. Set a new RFC3339 value, for example 2026-10-09T13:00:00Z. After a repository succeeds it stores that value, so the same value does not import it again. --force still imports the process you start immediately.
Stage and prod can import Maven through Pulp. Python and npm go through Tang, and Tang reads Pulp’s Postgres (clients.pulp.database), not the Pulp HTTP API, so those ecosystems are not available everywhere yet. Keep the app database.* settings on the local database. Do not point the app database at a stage or prod database.
resolves_cve_id matches an advisory when la.package_name = p.name and pv.version is one of fixed_versions. Maven advisories store package_name as group:artifact. The package row stores the artifact in name and the group in package_group, so that name comparison does not match Maven until the predicate changes. npm advisories store package_name as the purl name. The package row stores the scope in package_group (- when the package is unscoped) and the package name in name. Do not store the Maven coordinate or scope/name in name.
pv.version is the full stored string, including a rebuild suffix, so it can match fixed_versions that contain rhlw. The public list returns version as the upstream part and release as the suffix.
cve_id, resolves_cve_id, and vulnerable_to_cve_id match an element of lightwell_advisories.upstream. The advisory list is a few hundred rows, so this lookup has no GIN index. If it gets slow, add a GIN index on lightwell_advisories.upstream.
Zest is the Pulp HTTP client. Tang reads Pulp’s Postgres. These are the Lightwell reads that still go there.
Package catalog import (import-lightwell-packages) is the read that fills the mirror, so the mirror cannot replace it. Maven comes from Zest (ListMavenFlatPackages). Python and npm come from Tang (PythonPackageDetailList, NpmPackageList). Zest also resolves the repository and its latest version, so the job can skip a catalog fetch when that version is unchanged.
Repository import (pkg/external_repos/lightwell_repos_importer.go) asks Zest for the distribution URL (FindGenericDistributionByBasePath). The allowlist already has the name, content type, base path, and feature name. The mirror does not store that URL, and this import creates the repository rows the mirror uses. A missing distribution fails the import, so an unpublished repository is not recorded.
The coverage catalog (pkg/coverage/catalog/catalog.go) still loads from Pulp when the catalog source is pulp, which is the default. Java comes from Zest (ListMavenPackages) and Python from Tang (PythonPackageList). No client uses this path. It will be removed later.