| Proposal |
Summary
The current packaging of the Eclipse SDK as a set of large zips (one per platform) is a convenient one stop shop for getting up and running with the SDK. Unfortunately, this approach does not scale well. As more and more Eclipse projects (e.g., EMF, GEF, ...) come into common use, there is increasing interest in including these in pre-packaged zips. Similarly, as Eclipse is used in more varied scenarios, so increases the demand for different packagings (smaller and larger). It is not feasible for eclipse.org to create, manage and distribute pre-packaged configurations to meet all the demands. This proposal details a structure which enables Eclipse component providers to contribute their features and plug-ins and for Eclipse consumers to efficiently select, download and install their components of interest.Last Modified: November 22, 2004
The base Eclipse function is currently distributed as roughly the following zip files (as of 3.1M3)
Eclipse tools and technology project downloads are available separately and come in a variety of flavours, sizes and organizations. People wanting the base SDK are quite comfortable with the current situation. They need only download the single zip and go. If they need to run on two different platforms they download the zip for that platform and ignore the fact that 95% of the second download is the same as the first.
Users looking to combine the base Eclipse with additional plug-ins/features are somewhat less happy. They have to search over several web sites, identifying downloads (zips) of interest and correlating versions manually. In general, the suppliers of these additional zips do not make it easy for users to understand their offerings.
The main issues with the current setup are:
User starts with a minimal Eclipse base install. This contains just the runtime, some UI elements and an Update manager UI. Based on this they want to build an install containing additional elements from various sources.
After following the above scenario, the user can manage their configuration either using the steps outlined in Scenario 2 or by restarting their configuration with an indication that the update manager UI should be started. This puts them back at step 3 but with all their previously installed plug-ins still installed.
In this scenario, the user has an Eclipse product installed (possibly just the Eclipse SDK from a downloaded zip file or the configuration from the steps in scenario 1) and wants to add more function. This pretty much follows the normal update/install scenarios.
Note: This is an advanced scenario that may not be supported in the near term.
This scenario is very much like scenario 1. The main difference is in how the initial install is obtained. Rather than manually downloading and unzipping the minimal install, JNLP (Webstart) is used to fetch and run the minimal install.
We propose to use the Eclipse update technology to address the described problems and implement the listed scenarios. Note that this new structure is a complement to the existing large zip packagings of Eclipse. We are not proposing to stop creating the familiar SDK etc zips. (we leave it to someone more brave to propose that!)
eclipse.org (and others) will provide update sites sporting all the relevant features and plug-ins. Collections of projects can cooperate and share an update site or each supply/support their own update site. Managing your own eliminates the need for some sort of shared site update mechanism. However, the user must be able to navigate these individual sites seamlessly (see the section on Update manager).
There will be one update site per release family. A release family is a set of releases which form a substantial and separable effort. For example, Eclipse 2.0, 2.1, 3.0 and 3.1 are all different release families. The life of release family continues after its initial release (e.g., 3.0) to include maintenance releases (e.g., 3.0.1, 3.0.2, ...). Isolating release families to their own update sites helps maintain a separation between development going on for the next release family and maintenance work on the previous family.
Update sites are not possible unless we use the Eclipse version numbering scheme effectively. The main challenge is for plug-ins to have version numbers that accurately relate to their content. The essential element here is to ensure that if two plug-ins have the same id and version, their content is identical. The easiest way to do this is to use the version qualifier (i.e., the forth version element). For normal (not nightly) builds, the qualifier should be the version tag from the map file used for the build which generated them (e.g., 3.1.0.v20041122). The map file defines the content going into the build. If that does not change, then we can assume that the content of the output plug-in will not change. In this way, identical plug-ins will not be duplicated on the update site and consumers will not have to download them if they already have them. For more detail on plugin versioning, see the Plug-in Versioning Proposal.
Features on the other hand are collections of plug-ins. They are also relatively small. Since the content of a feature is indirectly defined by the content of the plug-ins included in the feature, it is hard to detect change. The safe bet here is to use the build stamp as the qualifier for feature version (e.g., 3.0.0.200402131200). This has the benificial effect of clearly identifying the features for a particular build. Further, since the entire feature set is regenerated, the collection of features with the same version number completely defines the entire build output.
Nightly builds do not go into an update site. There is no easy way to correlate the content in a nightly build (out of HEAD) with that of an integration build (using versioned resources). As a result, all plug-ins would be new and the economies of the proposed solution would not be realized. Further, nightly builds are primarily there for the individual teams to ensure they have not adversely affected other components etc. That is, they do not figure prominently in the scenarios driving this proposal.
Plug-in directories currently reflect the id and version of the plug-in. With the addition of the map stamp, the directory names will grow by typically ~10 chars). It is unlikely but this may push some path length limits. The use of the map stamp in a file/dir name may also place additional constraints the set of chars that can be used in a map stamp (i.e, CVS tag).
Sidenote on version qualifiers: The qualifier (fourth segment) is not interpreted. Qualifiers are compared using lexographical sorting. As such, care should be taken to ensure that the qualifier for subsequent builds monotonically increase. For example, if a build type identifier is to be included in the qualifier, it should be appended and the build date used as the main body (e.g., 20040217-I and 20040213-M7). See the complementary proposal on version numbering for more information.
The main challenges for the update manager are in making all of this scale and usable.
Interesting side note: using the runtime's new ability to run directly out
of jars, it is possible to have many different eclipse configurations sharing
the same actual plug-in files from the update manager's cache/store. This will
break some assumptions about plug-ins being together in a plug-ins directory etc
(the update reconciler may get upset) and it introduces some challenges for
cache clean up but it is an interesting possibility.
PDE Build already updates feature.xml files with the version numbers of the plug-ins involved in the build. To support the version numbering scheme outlined above, PDE Build will be updated to
Note: the plugin.xml updating outlined above must be carried out on any manifest.mf files present in the build.
PDE Build will also be updated to output directly to update site format. This is already largely supported but there maybe some modest work required.
A possible longer term advance is to change PDE build to only build the plug-ins which actually changed. Since we know the map stamp of the plug-ins we want in the output, we can first look for those in the update site (or some cache). If present, there is no need to rebuild them. This will have a huge (positive) impact on rebuilds during release cycles.
The problem outlined above is making it hard to scale Eclipse. It is hard for people to use large numbers of Eclipse components (e.g., EMF, GEF, ...) and we are failing to put forward a good model of how others would build/stock component systems. The ability to do this easily is key to the overall Eclipse component model. The steps proposed here are a good first try at solving the problem. The result will not be perfect but will allow us to understand what is wrong and fix it. Fortunately, the bulk of the technology required exists in Eclipse today. Also, since we are proposing a complementary approach to the current download strategy, we can stage the implementation of this new mechanism both as we have time and as we learn more.
The key steps that should be taken for 3.0 are