Eclipse JustJ Tools

Managing p2 Update Sites

To simplify and streamline the management of p2 update sites, JustJ provides the org.eclipse.justj.p2.manager application for the following purposes:

This infrastructure is used to generate the update sites for JustJ's JREs as well as the update site for JustJ's tools, including the JRE generator and the p2 manager tools. It is being exploited by a growing number of Eclipse projects 🔗. It can be directly used from within a pom.xml 🔗 to manage the updates sites as an integrated part of the build.

The Anatomy of a Managed Update Site

The following is the general folder structure of the set of p2 repositories managed by the org.eclipse.justj.p2.manager application.

2023-12 - Only present if -simrel-alias is used. It composes either nightly/latest, if that contains logical version 4.29.0, milestone/latest, if that is present and contains logical version 4.29.0, or release/4.29.0, if that is present. The logical version is determined by -version-iu or -version-iu-pattern release - Present if there is at least one release. It composes all the releases. milestone - Present if there is at least one milestone. It composes all the milestones. nightly - Present if there is at least one nightly. It composes all the nightlies.

Application Arguments for org.eclipse.justj.p2.manager

The org.eclipse.justj.p2.manager application recognizes the following command line arguments:

-quiet - Whether to avoid printing detailed logging information about the processing steps of the application.
-type - The type of the build.
-retain - The number of nightly builds to retain. Older nightly builds will automatically be deleted.
-timestamp - The timestamp, i.e., the exact time, of the build.
-build-url - The URL of the build job that produces the update site.
-commit - The URL representing the state of the Git repository used by the build that produces the p2 repository. This is stored as a repository property named commit.
-root - The root location of the local mirror of the update site. The promoted update site will be mirrored to this location. In addition, the -remote update site will be partially mirrored using rsync to this location. The index generator processes this location. The final result will be promoted to the -remote host also using rsync.
-target-url - The URL at which the -root of site will exist once promoted to the -remote server. This is used for the display of links and also for the generation of the p2.mirrorsURL property in the artifact metadata.
-relative - The relative location below the root at which to target the promotion and generation.
-remote - A remote source/destination specifification as used by rsync.🔗
-promote - The local filesystem location of the simple p2 update site to promote, or a URL for the remote p2 update site to promote. The promotion is done as specified by the -build-type. Note in particular that release builds do not promote the specified site.
-baseline-url - The URL from which to check for baseline replacements for the artifacts of the -promotesite.
-promote-products - The local filesystem folder location of the products, generally those produced by Tycho, to promote along with the update site. The product names are stored in the repository property products. This is generally used only for the update sites of a product. A download link for the each products will be available in the generated index.
-downloads - The local filesystem files of so-called downloads to promote and maintain along with the update site. These will be maintained in a subfolder called downloads nested within the update site. A download link for the each artfiact will be available in the generated index. The file names will be stored in the repository property downloads.
-version-iu - An ID prefix used to select or one more installable units with an ID with that prefix as the installable units whose largest version determines the logical version of the repository as a whole. Typically this a feature, particularly an SDK feature. If neither this nor -version-iu-pattern are specified, all installable will be considered which is likely to be inappropriate.
-version-iu-pattern - A regular expression used to select or one more installable units with a matching ID as the installable units whose largest version determines the logical version of the repository as a whole. If neither this nor -version-iu are specified, all installable will be considered which is likely to be inappropriate.
-iu-filter-pattern - A regular expression used to filter down the installable units, by ID, for which generated index information is to be produced. E.g., often one does not wish to list 3rd party installable units in the index.
-primary-iu-filter-pattern - A regular expression used to filter down the installable units, by ID, that are to be considered primary features, i.e., SDKs. This subset of primary features is presented on the root page and is shown in bold on all the other pages. If there are no primary features, then all features are presented on the root page. If all features are primary, then none of them are shown in bold on all the other pages.
-excluded-categories-pattern - A regular expression used to filter away category installable units with matching ID. Such removal helps reduce the likelihood that a user will install something from that category.
-maven-wrapped-mapping - Mappings for modifying or removing the maven-wrapped- (or maven-) installable unit properties, i.e., for mapping a maven coordinate groupId:artifactId:version to its replacement subsitution, or to nothing, for removal. E.g.,
org.eclipse.orbit:ant:(.*)
org.eclipse.orbit:(xyz:.*)->org.apache:$1
-label - The label used as the project name in the generated index pages.
-breadcrumb - An arbitrary label with optional associated URL, separated by a space from the label, for use in the breadcumbs of each generated index page. E.g.,
Justj https://eclipse.dev/justj
This argument may be repeated to specify multiple breadcumbs.
-archive - An arbitrary label with an associated URL, separated by a space from the label, for use in the archive navigation of each generated index page. E.g.,
0.0.1 - 0.10.12 https://example.org/downloads
This argument may be repeated to specify multiple archive links. The links are placed in a section labeled Archive immediately after the Release section in the navigation bar.
-favicon - A URL to an image that will be used as the favicon of each generated HTML index page.
-title-image - A URL to an image that will be used as the title image of each generated HTML index page.
-body-image - A URL to an image that will be used in the body of the generated HTML index page.
-mapping - A name mapping to convert a lower case name to a title case name. E.g.,
justj->JustJ
This is used to help produce improved navigation labels in the generated index pages. Without such a mapping, justj would map to Justj. This arugment may be repeated to specify multiple mappings.
-commit-mapping - A regular expresion mapping to convert a source commit URL to a target commit URL. E.g.,
(.*/)old-repo(.*)->$1new-repo$2
This argument may be repeated to specify multiple mappings.
--exclude - This argument and its associated value are passed directly as arguments to rsyn. This argument may be repeated to specify multiple folders to exclude from being transfered to or from the -remote host.
-summary - The number of update site columns in the summary table. Each row of the tabular summary corresonds to an bundle symbolic name of an installable unit with an ID matching the -summary-iu-pattern. Each column corresponds to an update site, starting with nightly/latest, milestone/latest, and then the releases from the most recent to the oldest, where the specified value for -summary truncates that list. Each cells displays the version(s) of the installable unit(s) with the corresonding bundle symbolic name in the corresponding site.
-summary-iu-pattern - A regular expression used to select the installable units with matching ID for display in the -summary table.
-simrel-alias - Can be used with a repository where the logical version of the repository, via -version-iu or -version-iu-pattern, can be mapped onto an Eclipse SimRel version, e.g., 4.30 maps to 2023-12. This option is used by Orbit to produce its update sites. The first nightly build that produces new SimRel version will result in the creation of repository named according to that version, e.g., 2023-12, which then composes that first nightly build. Each subsequent nightly build will be composed as the one child of that version repository, until the first milestone build. Then each subsequent milestone build will be composed as the one child of that version repository, until the release build, at which point the lifecycle is complete, and the verion repository composes the release. This provides a permanent and stable URL at the start of each release cycle which contains the latest content and then will ultimately contain the release content at the end of the release cycle.
-super - This is used by JustJ and is not generally intended to be reused, so don't use it.
-latestVersionOnly - Whether to mirror only the latest version or all versions when mirroring an update site. This is highly unlikely to be used.

Managing p2 Update Sites in Maven

The org.eclipse.justj.p2.manager application can be invoked as follows directly from a pom.xml:

  <properties>
    <eclipse.repo>https://download.eclipse.org/releases/latest</eclipse.repo>
    <justj.tools.repo>https://download.eclipse.org/justj/tools/updates/nightly/latest</justj.tools.repo>
    <org.eclipse.storage.user>genie.justj</org.eclipse.storage.user>
    <org.eclipse.justj.p2.manager.args>
      -remote ${org.eclipse.storage.user}@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/justj
    </org.eclipse.justj.p2.manager.args>
    <org.eclipse.justj.p2.manager.relative>tools-test/updates</org.eclipse.justj.p2.manager.relative>
    <maven.build.timestamp.format>yyyyMMddHHmm</maven.build.timestamp.format>
    <org.eclipse.justj.p2.manager.build.url>http://www.example.com/</org.eclipse.justj.p2.manager.build.url>
    <build.type>nightly</build.type>
  </properties>

  <build>
    <plugins>
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-eclipse-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <executionEnvironment>JavaSE-21</executionEnvironment>
          <dependencies>
            <dependency>
              <artifactId>org.eclipse.justj.p2</artifactId>
              <type>eclipse-plugin</type>
            </dependency>
            <dependency>
              <artifactId>org.apache.felix.scr</artifactId>
              <type>eclipse-plugin</type>
            </dependency>
          </dependencies>
          <repositories>
            <repository>
              <id>eclipse.repo</id>
              <layout>p2</layout>
              <url>${eclipse.repo}</url>
            </repository>
            <repository>
              <id>justj.tools.repo</id>
              <layout>p2</layout>
              <url>${justj.tools.repo}</url>
            </repository>
          </repositories>
        </configuration>
        <executions>
          <execution>
            <id>promote</id>
            <goals>
              <goal>eclipse-run</goal>
            </goals>
            <phase>generate-sources</phase>
            <configuration>
              <argLine></argLine>
              <appArgLine>
              <![CDATA[ 
                -consoleLog
                -application org.eclipse.justj.p2.manager
                -data @None
                -nosplash
                ${org.eclipse.justj.p2.manager.args}
                -retain 5
                -label "JustJ Tools"
                -build-url ${org.eclipse.justj.p2.manager.build.url}
                -root ${project.build.directory}/justj-sync
                -relative ${org.eclipse.justj.p2.manager.relative}
                -target-url https://download.eclipse.org/justj
                -promote ${project.basedir}/../../org.eclipse.justj.tools.site/target/repository
                -timestamp ${build.timestamp}
                -type ${build.type}
                -version-iu org.eclipse.justj.tools
                -commit https://github.com/eclipse-justj/justj.tools/commit/${git.commit}
                -breadcrumb "JustJ https://eclipse.dev/justj/?page=download"
                -favicon https://eclipse.dev/justj/justj_favicon.ico
                -title-image https://eclipse.dev/justj/justj_title.svg
                -body-image https://eclipse.dev/justj/justj.svg
              ]]>
              </appArgLine>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

Be aware that the procesing of the arguments does not properly handle tab characters as whitespace for separating arguments, so if you format the POM with tabs, it's best to use <![CDATA[ ]]> to ensure that the formatter does not insert tabs into your appArgLine.

Examples of Managed p2 Update Sites

The following is a sampling of update sites maintained by the org.eclipse.justj.p2.manager application.