Supporting Logical Models

The purpose of this document is to present details of the solutions for the scenarios presented in Enabling Logical Model Integration in the Eclipse Platform.

Logical Model Support Problem Categorization

The logical resource problem can be broken down into three areas that involve the Eclipse Platform. These areas can be summarized as follows:

1. The Eclipse Platform provides conceptually generic views that are Platform Resource model (i.e. IResource) specific.
• Navigator, Problems, Tasks, Bookmarks, Search
2. Tools, such as repository providers, that operate on the Platform Resource model should also be integrated with other models (logical to physical)
   • repository provider operations available on model elements that appear in the UI
   • while performing an operation, repository provider should show information in a form consistent with the originating model
3. Model tooling needs to be able to participate in certain Platform resource model changes in order to ensure model consistency (physical to logical)

The sections that follow explore each of these points in more detail.

Generic Views

The generic views problem is independent of the others problems in the sense that it can be solved without solving the other two and vica versa. The issue boils down to there being a view with a generic title, for example the Problems view, that only displays the problems from a single model (i.e. Platform Resources). This forces other models to either associate their problems with resources or create their own view. The main issue in generalizing these views is determining the best way to combine the problems, tasks or elements from multiple models in the same view. I can think of two ways to do this:

• show the problems, tasks or elements from all models in the same view.
  • This is the approach taken in the generic navigator proposal
  • Need grouping or filtering APIs to provide scalability
• provide a paged view where each page shows the problems, tasks or elements from a particular model
  • This is the approach taken by the Search view
  • Still needs grouping or filtering to deal with scalability

Each of these has their advantages and disadvantages. A single view frees the user from selecting which model to look at but such a view does not scale well. A paged view provides a means of top level filtering by model (advantage or disadvantage?) but can still have scalability issues.

The shape of the implementation of generic view support would depend on which approach to displaying multiple models was chosen and the nature of the view. The shape of the solutions in the area would really need to be driven by the product teams either in the form of scenarios and workflows or prototypes (or both).

Problems View

This view seems to be of most interest to modeling tools built on top of the Platform since it deals with helping the user correct model inconsistencies. For this reason, we will outline the beginnings of a possible solution here.

Based on several discussions on this topic, the following is a list of the desired properties of a generic Problems view.

• All the problems appear in a single view (i.e. not multiple pages) but are not tied to resource markers
  • Requires a generic problem API
• Support filtering
  • include/exclude problems from different domains
  • by severity
  • domain specific
  • selection
• Support grouping
  • arrange problems hierarchically. e.g. problems grouped by logical or physical resource
• Sorting
  • sorting across domains
    • severity
    • description
  • domain specific
• Actions
  • Goto action, Show in and QuickFix support that performs an action appropriate for the selected problem
• Efficient addition and removal of problems
  • problems may be removed/added one at a time or a domain at a time
• Display information appropriate for problem type
  • generic columns (severity, description, resource, path, location in resource)
  • allow user configuration of displayed columns (included domain specific columns)
  • support separate pane that shows details of selected problem or alternate views of the problem (i.e. problem dependency graph)

Search View

The search view already has support for searches from different model domains. This comes in two forms:

• The Platform has pluggable support for search types. Some example search types are File Search, Java Search, Plugin Search and NLS Search. Each search type, when executed, adds a custom search result page to the search view.
• Some search types support participation. For instance the Java Search allows particpants to be added to Java searches.

Generic Navigator

There is work in the WTP project on a generic navigator and the rumor is that this may get pushed down to the Platform sometime after 3.1.

Bookmarks View

This is similar to the Problems view but is probably a bit simpler. Solutions to the problems view apply to this view as well.

Tasks View

The Tasks view is a bit more complicated. Currently, tasks are tied to resources just like problems and bookmarks so a similar solution could be applied to the Tasks view. However, one could easily argue that tasks could be associated with multiple resources (or no resources). This makes generalizing Task related concepts more difficult than that of problems and bookmarks.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=6904

Logical to Physical

A well integrated development environment should support 3rd party operations, such as repository provider operations, on UI model elements that map to the underlying Platform Resource model. Eclipse currently does support this for cases where there is a one-to-one mapping from a model element to a resource. However, to get full integration, one-to-many, and many-to-one should also be supported.

Performing Resource Based Operations on Logical Models

In order for resource based operations, such as repository operations, to be performed on logical models, a means to communicate resources that constitute the model elements is required. This is the purpose of the ResourceMapping API introduced in Eclipse 3.1 M5. This mapping supports a "one model element to many resource" mapping. The specifics of this support is outlined here.

The following points summarize how ResourceMappings are used to communicate the resources of a model element to a repository provider.

1. Repository providers contribute menu operations to model elements through the Platform adaptable mechanism. The menu items will appear on any elements that adapt to ResourceMapping.
2. When the user performs a repository operation on a model element, the contributed operation is handed the ResourceMappings obtained from the model elements.
3. What happens next depends on what operation was chosen. For instance, a check-in (or commit) operation can use the resource mapping to check-in the appropriate files. Other operations can similarly be performed on the resources that constitute the model element.
4. In some cases, the remote state of the model may contain additional resources. The ResourceMappingContext API is a way to allow the repository provider to communicate to the model provider the means to determine the remote state of the model so that all relevant resources can be included in the operation. For instance, an html file model may include the images shown on a page. If additional images are added to the remote state, operations such as update or merge would want to include the new images.

Once an operation is triggered, some operations may need to obtain additional input from the user or show operation feedback. Ideally, the information shown in dialogs or other views used to obtain information from or show information to the user should be consistent with the model from which the operation was launched or, in the very least, provide a link between the resources being operated on and the originating model elements. The following list outlines some of the different scenarios where this issue may occur:

• It may be desirable to provide the ability to exclude some of the model elements from the operation due to resource state (e.g. may want to prompt when replacing model elements that contain dirty files). When a single model element maps to one or more resources, this is straight forward. However, if many model elements are contained in the same file, the model elements may need to be included/excluded as a group. This is further complicated by the fact that all affected model elements may not be contained in the input to the operation. This is covered in the "Handling Many-to-One Mappings" section.
• Repository operations often involve the merging of remote changes with local changes. This involves fairly complicated interactions between the model and the repository. This discussed in more detail in the next section.

Merging Logical Model Changes

One of the more complicated scenarios involving repository providers is when remote changes need to be merged with local changes. Here are the merge cases:

• When a merge is required, a headless merge may be possible at the file level which means that it could be performed by the repository provider without the involvement of the model (i.e. no knowledge of model semantics is needed).
• It is also possible that a headless merge is not possible at the file level but is possible at the model level (i.e. the knowledge of the models semantics makes a headless merge possible).
• If a headless merge is not possible, the user will need to perform a manual merge.

The interesting cases are the last two (i.e. merge involving model semantics and manual merge).

There are several complications that must be considered when formulating a solution:

• The elements being merged may come from different model tooling and hence are parts of different models. This is currently unlikely since there are few generic views showing elements from different models but may become more likely in the future as more generic views surface.
• At the lowest level, merges happen on files. If the multiple model elements are contained in a single file, it is highly possible that additional model elements may be affected by the merge. This means that the scope of the merge may be greater than what was originally indicated by the user when they launched the operation that triggered the merge.
• The model elements being merged may be contained by higher level models or may involve files that are contained by other models. The challenge here is to display the merge in a way that makes this obvious to the user and allows them to decide which merge mechanism is more appropriate. In the case of layered models, it may be appropriate to choose the highest level model as the default merge mechanism.
• In order for model tooling to display the merge in a form that is consistent with the model, access to the ancestor and remote state for files needs to be accessible.This would require API provided by the repository provider.
• The merge happens as part of a repository operation. When the merge is complete, the repository operation may then need to perform additional portions of the operation (e.g. modify the synchronization state of local resources). This is complicated by the fact that the merge may have been completed on a subset of the elements included in the operation (and the other elements skipped).

A solution for presenting model based merges could take several forms:

• A separate dialog or wizard page for each model type involved in the merge. This is OK if the models are disjoint but may be a problem if there is overlap.
• A single page that displays all the model elements being merged. Selecting a model element will populate other portions of the page appropriately. The advantage is that the user will see the entire scope of the operation at once. However, it will still be difficult to highlight model overlap. This will also require a more standardized presentation which may be more restrictive.

The second approach is already supported by the compare framework and feels like a better approach. Hence, we will concentrate on that solution for the time being. Here is what this solution may involve:

• A mechanism is provided for obtaining a model specific DiffNode (Compare component API) for each ResourceMapping that is being operated on. This would allow the merge operation to convert the selection into a form that could appear in a merge viewer. Each type of DiffNode may have model specific operations that should appear in the context menu or toolbar.
  • The mechanism for creating the DiffNodes will need access to the remote and ancestor (or base) state of the model.
  • Need to be able to include additional DiffNodes from a model for the case where one of the selected elements is contained in a file with other model elements.
  • Ideally, DiffNodes could be obtained from other models that are interested in the resources being merged.This becomes more complicated as there may be relationships between multiple models (i.e. one may be built on top of another) and you would want the display of the DiffNodes to reflect this.
• A merge viewer that can contain DiffNodes from one or more models. Operations appropriate to a selected node should appear in the context menu and toolbar. Also, selecting a node should update the other portions of the merge viewer (structured comparison and content comparison panes). Expansion of DiffNodes should also be supported.

The key here is that the model is in complete control of the merge of its model elements (i.e. the user may not even need to be aware of the underlying files).

Handling Many-to-One Mappings

Another interesting case involves the case where a single Platform resource contains several model elements. If the user selects a single model element and performs a repository operation on it, it will affect other model elements as well. This will be confusing to the user if they see other model elements being affected by an operation on one element. Here are some the scenarios involving this case:

• The user wants to tag (i.e. label or version) a specific model element but doing so will tag all the elements contained in the same file. It may be enough to warn the user that more than one element will be tagged but ideally, the user could be shown all the elements that will be tagged.
• The user wants to merge a model element. If the model provides the merge viewer, than this may be possible without involving the other elements. However, the repository provider would need to be able to differentiate this from a full file merge since the action taken upon merge completion may differ. For some operations that start with a merge, such as check-in, all the elements in the file would need to be merged.
• The user wants to replace a model element with the contents from the repository. The operation doesn't need a merge since local contents are being replaced. However, like a merge, it may be possible for the model to perform the replace on the specific element given the contents of the two files. Without this capability, the user would need to be informed that additional elements will be included in the operation.
• Goto action on a file marker (https://bugs.eclipse.org/bugs/show_bug.cgi?id=87288). If many elements are contained in a single file, it is possible that multiple editors may be open on the same file. A mechanism to pcik the proper editor is needed.

Possible solutions to this problem are:

• Provide a hint on the resource mapping or traversals that the file is not wholly contained by the model element. This would at least allow the repository provider to warn the user that the operation will affect additional elements. If the traversal also included the range within a file, this could be used for linking operation such as the one involving the linking of markers to editors mentioned above. This could also be used to perform replacement of file sections but would require the the ability to obtain the corresponding range from the remote contents as well.
• Provide a means of obtaining the additional model elements that will be affected. This will allow the repository provider to display all the elements that will be affected but requires additional API.

This problem has some overlap with the physical to logical problem which is covered in the "Physical to Logical" section, in the sense that, if a mechanism existed for finding all model elements from any model that are contained within a resource, this could be used as the API for the second point mentioned above.

Decorations

Decorating logical elements based on their resource mapping is problematic is some cases. The most interesting case is decorating elements to indicate that they have changed locally with respect to the repository. There are two aspects of this that are problematic:

1. When a file contains many logical elements, only a few may be dirty when the file is dirty. However, they will all appear dirty if the file dirty state is used to determine the decoration. This can be overcome by allowing the model provider to determine model element dirty state by comparing the modified local file contents with that of the unmodified contents from the repository. Unfortunately, this requires that either the unmodified contents are cached or fetched from the server.
2. When elements are shown in a tree, the resource mapping of a parent may not contain all the resource mappings of its children. This is due to the use of content providers in tree views that determine what the children of an element are in the tree. The only solution to this is to use model elements that contain the logic of determining their children and use a simple content provider that obtains the children from the model element. The problem with this approach is that it leads to model wrapping when alternate views are used (which is something the content provider API was supposed to solve).

Physical to Logical

In Eclipse, there can typically be many different model spaces. Many of these can be considered layers on top of the Platform resource model, in the sense that the elements that constitute a model are persisted in Platform resources. When operations are launched from model elements or are performed by model tooling, consistency of the model can be maintained. However, given the presence of multiple models and multiple layers, it is always possible that operations are launched from elsewhere (i.e. the tooling of ather models) and are not aware of all the models that could be affected. For instance, deleting a file from the Resource Navigator may corrupt a higher level model that was persisted, at least partially, in that file. In this case, all the model tooling can do is make a best effort at cleaning up and notify the user of any inconsistencies that may exist.

It would be beneficial if model tooling could participate in operations that could potentially corrupt a model. This participation could take many forms:

• receive pre-notification of an operation that is about to be performed
  • notify the user that an operation they are performing may corrupt a model
  • abort an operation that would be too destructive
• participate in the operation itself
  • augment the operation is such a way that model consistency is maintained
• perform post-operation processing to adjust the model to the change

There are several complicating factors:

1. The set of resource operations that can corrupt a model needs to be identified.
• setting file contents and moving or deleting files/folders definitely may corrupt a model
• what about creating or copying resources?
• project close
2. At what level should the participation take place.
• Higher levels provide more context but are more numerous. Basically, each layer would need to provide a participation API for higher layers to use.
• Lower levels are fewer but may not give the entire context of the operation (or may not have direct access to a UI context)
• Even if higher level participation is supported, you would want lower level hooks
  • in case some tooling doesn't provide participation
  • two independent models are built on the same resource (although this seems unlikely to occur)
3. There may be multiple models interested in a single resource.
   • Determining which models are interested must be efficiently computed.
   • The case of multiple interested parties, although potentially rare, needs to be handled in a reasonable way
4. Augmenting an operation may result in further operations that again need to go through the participation process

It should be noted that the support required for physical-to-logical could potentially be used to help in the many-to-one mapping case as well.

Status

Here is the status of this work as of 3.1 M5

• Generic Views: This needs to be driven by model owners with help from Platform committers. Until model teams step forward and dedicate resources to this problem, it is unlikely that work will progress.
• ResourceMapping/ResourceMappingContext: Initial cut is available in 3.1 M5 and will be in 3.1 as provisional API.
• Merge Support: Although work has not begun, this area is well bounded and well understood.
• Physical to Logical: This area is not yet well understood or well bounded as participation in operations can happen at multiple levels and involve multiple models. More design work is required before a solution can be proposed.

Open Issues

Here are the list of open issues:

• Transaction support: Some repository may have transaction support. It is unclear how this fits in with logical model support.