Exercise 06: Metadata

What This Exercise Is About

Among other factors, good project health comes from the abililty to quickly audit and track changes. For developers, it's handy to know what was changed for a given bug in case you need to backport those changes to an older branch. For users, providing this information can give the knowledge and satisfaction that their bug or patch was applied, which will in turn encourage them to provide more patches.

What You Should Be Able To Do

At the end of the lab, you should be able to:

Required Materials

Exercise Instructions

This exercise is carried out entirely using the Eclipse Software Development Kit (SDK) version 3.4M5 with the Eclipse Modeling Framework (EMF) 2.4M5 installed into it. The exercise instructions refer to this product as either Eclipse or as "the workbench."

These instructions are located in your workspace, in the Build_Workshop/Exercise06_Metadata folder.

Directions

Step A: Setup

  1. Switch to the Plug-in Development perspective, if not already there.
    1. Select Window -> Open Perspective -> Other... -> Plug-in Development.
  2. In the Package Explorer view, right-click the file Build_Workshop\Exercise06_Metadata\exercise_06.psf and select Import Project Set.... If prompted to overwrite, click Yes To All.
  3. If you prefer, you can also use the Import -> Existing Projects into Workspace method from Exercises 01 and 02, using Solution06.zip.

Step B: Review project metadata

There are numerous ways to introspect your project. In this exercise, we will explore seven.

  1. When a build is published, an RSS feed is created or updated, storing information about that build such as version, buildID, and cvs branch. 

      <entry>
    <title>[announce] jet 0.8.2 (R200802201614) is available</title>
    <link href="http://www.eclipse.org/modeling/m2t/downloads/?project=jet&showAll=1&hlbuild=R200802201614#R200802201614"/>
    <id>http://www.eclipse.org/modeling/m2t/downloads/?project=jet&showAll=1&hlbuild=R200802201614#R200802201614</id>
    <updated>2008-02-28T16:17:44Z</updated>
    <summary>
      <build xmlns="http://www.eclipse.org/2006/BuildFeed" branch="R0_8_maintenance" datetime="200802201614" href="http://www.eclipse.org/modeling/m2t/downloads/?project=jet&showAll=1&hlbuild=R200802201614#R200802201614" type="R" version="0.8.2">
        <detail config="build.cfg" href="http://www.eclipse.org/downloads/download.php?r=1&file=/modeling/m2t/jet/downloads/drops/0.8.2/R200802201614/" log="buildlog.txt" map="directory.txt"/>
        <update>http://download.eclipse.org/modeling/m2t/updates/</update>
        <downloads>http://www.eclipse.org/modeling/m2t/downloads/?project=jet</downloads>
        <releasenotes>http://www.eclipse.org/modeling/m2t/news/relnotes.php?project=jet&version=R200802201614
          </releasenotes>
        <releases>
          <release arch="ALL" os="ALL" type="SDK" ws="ALL">m2t-jet-SDK-incubation-0.8.2.zip</release>
        </releases>
        <tests>
          <test href="http://www.eclipse.org/downloads/download.php?r=1&file=/modeling/m2t/jet/downloads/drops/0.8.2/R200802201614/testResults.php" type="junit">
            <result arch="ALL" os="ALL" ws="ALL">PASS</result>
          </test>
        </tests>
        <dependencies>
          <dependency>http://download.eclipse.org/eclipse/downloads/drops/R-3.3.2-200802211800/eclipse-SDK-3.3.2-linux-gtk.tar.gz</dependency>
          <dependency>http://download.eclipse.org/modeling/emf/emf/downloads/drops/2.3.2/R200802051830/emf-sdo-xsd-SDK-2.3.2.zip</dependency>
        </dependencies>
        <coordinated status=""/>
      </build>
    </summary>
  </entry>

  2. This information is then used to add releases into the database that backs the project's release notes. 

    mysql> select * from releases where component like '%jet' and buildtime > '2008-02-01' order by buildtime desc;
+-----------------+-----------------+---------------+---------------------+------------------+------+
| project         | component       | vanityname    | buildtime           | branch           | type |
+-----------------+-----------------+---------------+---------------------+------------------+------+
| org.eclipse.m2t | org.eclipse.jet | I200803111920 | 2008-03-11 19:20:00 | HEAD             | I    |
| org.eclipse.m2t | org.eclipse.jet | I200802261900 | 2008-02-26 19:00:00 | HEAD             | I    |
| org.eclipse.m2t | org.eclipse.jet | 0.8.2         | 2008-02-20 16:14:00 | R0_8_maintenance | R    |
| org.eclipse.m2t | org.eclipse.jet | 0.8.2RC       | 2008-02-20 16:14:00 | R0_8_maintenance | S    |
| org.eclipse.m2t | org.eclipse.jet | 0.9.0M5       | 2008-02-12 17:45:00 | HEAD             | S    |
| org.eclipse.m2t | org.eclipse.jet | I200802071120 | 2008-02-07 11:20:00 | HEAD             | I    |
| org.eclipse.m2t | org.eclipse.jet | I200802052310 | 2008-02-05 23:10:00 | HEAD             | I    |
| org.eclipse.m2t | org.eclipse.jet | M200802052255 | 2008-02-05 22:55:00 | R0_8_maintenance | M    |

  3. This data is then in turn queried and turned into filterable release notes. Compare the following links:


    4. JET release notes: note the filter options & query tools/links

  4. In addition to the Release Notes, there is another query tool -- Search CVS. This provides a view of the latest commits to the project, indexed by a number of filterable parameters. In the above screenshot, clicking the image or entering a bug number into the query field in the top right will query the database for the changes associated with that bug. You can also query by:


    Search CVS sample query

  5. To see the specific change associated with a given CVS delta:
    1. Choose an interesting build, such as 0.9.0M5.
    2. Choose an interesting bug, such as 196814. Click the delta link.
    3. Choose an interesting change, such as the most recent change in createBasicProject.jet. Click the 1.4 > 1.3 link to launch ViewVC and generate a CVS diff.

     

    ViewVC sample CVS diff

  6. There is also a tool to generate a changeset for a given bug. It is linked on both the Release Notes and Search CVS pages. The tool will create a bash script which can be run to generate a patch file.
  7. Finally, there is another application of the release notes database: Build News. Like the RSS feed, this provides a snapshot view into the project's recent releases.


    Located on the right-hand side of the M2T homepage

Step C: Updating the database / staying current

  1. To update the database's tables, loading in new bugs, bugdescs, releases, and commits, there is a convenient web API available:
    1. Update Search CVS
  2. This API is used during builds and promotes to ensure that the database is current for that project.
  3. Nightly updates are also run via crontab for all projects. 
    00 16 * * * /opt/public/modeling/searchcvs/parsecvs.sh

Step D: Linking Bugzilla to CVS

  1. To identify that a given commit is related to a given bug, developers need to label the commit with the bug number. Numerous formats are supported (bug 153838, bug 164719), but the recommended one is to simply use: 
    [164719]
  2. If you use Mylyn, set Window -> Preferences... -> Mylyn -> Team -> Commit Comment Template to [${task.id}].
  3. If not, just ensure that every commit that is associated with a bug has a comment which begins with a bug number in square braces, and an optional comment thereafter.
  4. Should a commit be relevant for more than one bug, simply enter one bug number & optional comment per line of commit comment.
  5. That's it!

Step E [Optional]: Explore code

  1. The code for generating RSS feeds and using the Search CVS database is all publicly available @ Eclipse.org.
  2. If you imported the project change set in Step A, you will have a new project, org.eclipse.build.tools.
    1. Explore RSS feed generation in:
      • org.eclipse.build.tools/src_rss (java source),
      • org.eclipse.build.tools/scripts_rss (shell & ant scripts),
      • org.eclipse.build.tools/data (sample data),
      • org.eclipse.build.tools/properties (sample properties files),
      • org.eclipse.build.tools/model (EMF Ecore data model) and
      • org.eclipse.build.tools/schema (XML Schema data model)
    2. Explore Search CVS database admin tools in:
      • org.eclipse.build.tools/scripts_cvs/searchcvs

Summary

You have learned how to use metadata to explore a project's activity, for tracking, auditing, and querying. For further reading, see: