Summary
The goal is to provide a basic and extensible way of sharing projects across workspaces. In this context, sharing does not mean to have the same physical files used by different workspaces. But some way of transporting/copying projects through workspaces.

By Rodrigo Peretti, OTI
Last Modified: December 06, 2001

Motivation

Proposed Solutions:

    Consists of storing one or more projects in a file containing the project's resources and metadata information specified by the user. The file is written when an export operation is requested and read with an import operation. The proposed solution is in API format and would be added to IWorkspace.

Constants:

• METADATA_ALL
• METADATA_PROJECT_DESCRIPTION

• availableProjects
• exportProjects
• importProjects

Comments about this approach ( - favour  against):

1.1 - Why should we have this API if we already have API for accessing metadata? Isn't it just a convenience API?

1.1.1 ( - ) For now we do not have any plans on making all the metadata available through API. For example, things like the "resource tree" and the various information bits held on "ResourceInfos" might never become API. The export/import mechanism can be a way for sharing this metadata through workspaces without committing to undesired API. But, as you point out, the first implementation of the proposed API could be done with existing API and we do not know if it is going to evolve to what I pointed out on the previous sentence. It could end up as only a convenience API.

1.1.2 ( + ) It seems to me that this is the direction (metadata via a simple api) that would be useful for the cut/copy/paste case. And as Greg suggests a simple UI change to include/detect this metadata on export/import means that project export/import could happen with only minor changes to to the UI.

1.2 - Why should the UI or even the user be responsible for cleaning up the mess if the operation fails? As a user I would just go and delete it all and start again. I would be very nervous about what actually got copied.

1.2.1 ( - ) Here is an extreme use case: suppose the export file is corrupted but anything you can recover from that is worth a try. If core or UI delete things when something goes wrong, the user does not have a chance to recover these pieces of data.

1.2.1.1 ( - ) I think the ability to show the user a yes/yes to all/no/cancel dialog when an overwrite situation occurs is a requirement (see IOverwriteQuery). We are able to perform this for current import/export because we are performing them one resource at a time. Without a callback from core for overwrite during project import we will not be able to make project import work like file system/zip import.

1.3 - It seems that I can set overwrite or force to true but then if a collision happens the operation just stops and reports that error potentially leaving things half done. This would require the UI cleanup (although core should do that), and then if the user says "yes go ahead" the UI would have to start it all from the begging with force/overwrite being true. Typically when an overrwite happens there is a yes, yes all type confirmation given however there does not appear to be sufficient support to provide this?

1.3.1 ( - ) True, there is no support for that as there is no support for it on similar APIs like IResource.move and IResource.copy. It seems consistent through the resources API but there's room for discussion.

1.4 - In summary it is unclear to me why core would want to get into creating a projectExportFile. It seems to be just a zip that includes some metadata. Why doesn't core simply add api to produce/read this metadata description? A file with this metadata could be added to the project/read from the project, on export/import with only minor modifications to existing File System and Zip wizards.

1.4.1 ( + ) Randy's suggestion is an interesting variant on my observation. I was observing that if the meta data was just always stored as a file then no special facility/api is needed. Randy's interesting variant says if we really have to store it not as a file, then all we need is the ability to drop it into a file on export, and to create a project using the file (which we could call on import).

1.4.2 ( - ) It seems that an approach having a separate file holding the metadata dropped into the project content area can be brittle. Seems very easy for this file to get out-of-date. Having a single file containing resources + metadata sounds more consistent.
 

1.4.2.1 ( - ) If I understand the purpose, the metadata file shouldn't appear as part of the project in the workspace after an import.  The only way it could go out of date is if the user unzipped it, modified some stuff, and rezipped it.
 
1.4.2.1.1 ( + ) Agreed. It was a misundertanding. It would only be a problem if the file stayed around after an import. The callers of this possible API would have to take care of deleting this metadata file.

1.5 - I'll confess to not being sold on the one big file.  It feels like we are inventing a new format, when we have existing export/import that work. In the past we've usually had  grief when using special files, and we sometimes find we have to keep adding & adding api as we want to add features. For example, assume we ever wanted to piecewise load folders from the file, we would need special api for it. But if it was a standard import/export format it would be doable without change.  If we wanted to show a user a view showing them the various available exported wads and the detailed contents of each, we can't do it without additional api (current spec'd api only gives us available projects).

1.5.1 ( - ) The proposed API is flexible enough in order to add more metadata to the export file. About other features, yes, we would have to add more API.

1.6 - In the big-file case I am not sure what the spec'd behavior is if you hit a corruption. Do you stop immediately or continue just skipping the bad things. If you stop immediately then it would be limited in what I can recover. Things after the corruption go unrecoverable. This seems to suggest we would be safer to avoid a special big-file format. If it was standard export with meta flattened into it then a user could recover parts as needed.

1.7 - It is not clear to me that this proposal can support cut/copy/paste. Transfers (the classes which implement the code required for cut/copy/paste) are stateless. Any information they require must be placed on the clipboard. This can be real data as in the case of a text transfer or a reference (ex. filename) as in the case of a file transfer.
    Performing project copy/paste within a workspace is easy. We just place a reference (path) for the project on the clipboard and then lookup the real project (to copy it) when we paste.
    In order to perform a cut/copy/paste across workspaces we need the metadata required to recreate a project in the second workspace. We don't need the actual contents of the project since we can just use a reference to the location of these contents.
    The alternative (given the proposed api) is that cut/copy would create a temporary projectExportFile and place a reference to it on the clipboard. However this violates the principle of not doing any real work until paste (if you were just "copying" the project to get its name to paste it into a text editor then creating the export file is a real waste) .
    A comparison I would make is that VAJ had the abiltiy to create a jar or just create a manifest. Core should have api to create a projectExportFile (data + metadata) or just just the project metadata (so that it can be placed on the clipboard along with a reference to the data).

Solution #2 - Have the shareable metadata under the project's content location

    Consists of having all the shareable information available under the project content location (like it already happens with .vcm_meta and .classpath).

Comments about this approach ( - favour  against):
 

2.1 - The VCM plugin exports/imports projects, but not into a zip/jar but into a repository. There is one main undocumented requirement that VCM has imposed on plugins that want do be VCM import/export compatible: a plugin _must_ save shared configuration information about a project into the project's file hierarchy and not in .metadata (e.g. .classpath is managed by JDT). However, the resources plugin does not play by the same rules. The VCM plugin was made responsible for persisting the shared configuration information for the resources plugin (e.g. .vcm_meta).

2.2 - Why should the resources plugin be special in regards to VCM import/export requirements?

2.2.1 ( - ) I agree that it should not. But it is easy to look at the current scenario and say so. Currently, the only resources information (that I know) VCM is sharing in the project content area is the project description. This information can be easily represented in one file and changes to this file do not need to be merged. Now, thinking about the future and getting something more complex like markers (or persistent properties). As an implementation detail, they are currently persisted as one unique file. If we keep this format, this file would need to be merged if it gets out of sync with the repository. To do so, more APIs will have to be added in order to VCM inspect it and figure out what has changed and ask the user what should be done (as it currently does with text files). I'm not saying it is impossible, only that it is a kind of change that will likely be hard to happen.

2.3 - The resources plugin is not the only plugin interested in importing and exporting projects. Eclipse just don't have a consistent story for how plugins can identify shareable configuration information. The suggested API is, in my opinion, another way of avoiding the issue of how to handle the problem in a more general way. So how can the general import/export mechanism support both VCM's and a zip/jar tool needs?

2.3.1 ( + ) Agreed that Eclipse should have a consistent story on how plug-ins can identify shareable config info. Here's a first try proposal. It is a mix of what has already been discussed. We do need feedback from everyone specially the VCM and UI in order to understand if it meet their needs.
Basically, a .metadata folder will exist under the content area of each project. It will hold all (or most of) the metadata provided by the resources plug-in plus any other metadata (like the .classpath file) provided by other plug-ins.
The interesting point is that some of these metadata will not (usually) make sense to be shareable via VCM but (usually) makes sense when a single user wants to move/copy a project from one workspace to another. Clear examples are markers and persistent properties. The metadata file that holds the markers cannot be merged. Thus, cannot be safelly used against a CVS repository by multiple users. Although, a user that wants to move a project to a new workspace and preserve bookmarks, tasks and breakpoints, etc... does want to move the markers metadata file.
What this approach requires from plugins (first cut, need more feedback):
* Resources: move all the metadata info to the new .metadata folder under the project content area. Provide a default state (plug into some VCM extension point) for each of these metadata files indicating which ones are appropriate to be shared in a team environment.
* VCM: It is already there but just to highlight it here. A way for plug-ins to say what files (or kind of files) should be ignored when dealing with the repository and make it possible for users to change this configuration as they need.
* UI: For the export wizards, the functionality might already be there. Also, the user should be able to point to a folder (project folder) or to a file (.prj file) and choose "create existing project from folder/file".

RFC Status