Problem Definition - Inflexible Project Structure

Summary
Eclipse requires that the contents of each project be stored in a single directory on disk. Every file and folder in that directory tree on disk must belong to the project in the workspace. This restriction makes it difficult to use Eclipse in conjunction with tools that have specific layout requirements for their files, or with users who have legacy file base layouts that they need to maintain. We would like to improve the situation for the 2.1 release of Eclipse. This, the first of two documents, describes the nature of the problem; possible solutions are discussed in a follow-on document.

Last Modified: 14:00 October 4, 2002

A number of users of the Eclipse Java IDE complain that Eclipse is overly constraining in where their files are located. Before looking into these complaints, we quickly review how projects are structured in Eclipse (both 2.0 and 1.0).

Review: Properties of Eclipse project structure

Here are the properties of Eclipse workspace projects:

A workspace consists of any number of projects.
A workspace has a root directory on disk.
The workspace data area holds metadata, and possibly project files.
Each project has a root directory on disk.
All of a project's files are contained under the root directory.
All files under the root directory show up in the project.
A project's root directory has a default location within the workspace directory, but can also be assigned to an arbitrary location on disk
A project's root directory cannot coincide or overlap with any other project's.
Each project can have a team provider.
A project descriptor file (".project") appears in the root directory of each project.

The Java development tools uses projects in the following way:

A Java project can have one or more source folders on the compile-time class path.
Each source folder contains Java source files and other resources in a Java package hierarchy.
A source folder must be a subdirectory under the Java project (at any depth).
A source folder must not coincide with or overlap any other source folder.
A Java project has one output folder.
The output folder must be a subdirectory under the Java project (at any depth) to hold generated class files (and copied resource files).
The output folder may coincide with a source folder, but may not be nested within it; and conversely.
A Java project can have any number of library JARs on the compile-time class path.
A library JAR may be anywhere within the Java project, within a different project, or outside the workspace entirely.
A Java project can have any number of other Java projects on the compile-time class path.

Complaints about lack of flexibility in Eclipse project structure

The following complaints are representative, but not exhaustive. See References for the related bug reports and newsgroup postings.

Complaint 1: Unable to locate Java output folder elsewhere

There is no way to locate a Java project's output folder other than somewhere under the project subdirectory. There are several different reasons people complain this is too constraining:

It means that generated class files cannot be squirted into a particular location relative to other files. For example, the class files need to be in specific location, such as C:/tomcat/webapps/mytest/WEB-INF/classes/, so that they are seen by a test web server. It is not feasible to locate all the files for the Java project within the test server's directory. This means a separate step is required to copy the files from the Java project output folder to the test server's directory.
It can lead to poor performance. The most important files in the project are the source files, so it makes sense to locate these files on a network drive that is backed up regularly. Locating the project's root directory on a network drive provides a developer with protection against losing important work. Unfortunately, this means that the class files in the output folder also end up being stored on that network drive. Many class files are deleted and rewritten each build, leading to increased network load and slower compile times. Performance would be better if the output folder could be put on a local disk drive, separate from irreplaceable project files.

A related complaint is that there is only a single output folder per Java project. It would be more flexible if it was also possible to associate output folders with each Java source folder.

Complaint 2: Unable to deal with Java source files laid out in pre-ordained structures

Many complaints begin with a statement like: "At my company, the directories are structured in a certain way that can't be changed." A pre-ordained layout for source files can come from any number of real-life considerations. The most common one is that many companies have a large pre-existing source code base stored in a company-wide code repository; changing to a different layout would be a very expensive undertaking.

As the following examples illustrate, Eclipse cannot deal directly with source files laid out in many pre-ordained structures. This means that using Eclipse requires developers to work instead with a copied and rearranged source file base.

Example 1:

The source files for the company's products are laid out in one big directory that is version and configuration managed outside Eclipse:

If a Java project is created to work on the files in the Product1 directory, the project's output folder and Eclipse metadata files (.classpath and .project), will end up somewhere under AllProducts. This pollutes the source tree with files that do not belong there.

Example 2:

The Java source files for the company's products are all held in a single package directory like this:

If a Java project is created to work on Product 1, AllJavaSourceFiles will need to be specified as a source folder. But this is unworkable because it means that all the source files for Product 2 will also be included in the Java project and will get compiled along with those of Product 1.

Example 3:

The Java source files for a product are laid out in package directory like this:

If a Java project is created to work on Product 1, both Product1/JavaSourcesFiles and Product1/JavaSourcesFiles/tests would need to be given as source folders. Eclipse does not allow this (source folder are not allowed to overlap). If Product1/JavaSourcesFiles/tests is not declared as a source folder, then the Eclipse compiler expects P1Test.java to be in an incorrect package ("tests.com.xyz.product1.tests" rather than ("com.xyz.product1.tests"). Putting the tests in a separate Java project does not work either.

Example 4:

The Java source files for two products require a common framework:

If a Java project is created to work on Product 1, both Product1/JavaSourcesFiles and CommonFramework/JavaSourcesFiles need to be specified as source folders so that all the code needed for Product 1 is compiled. This means that CommonFramework/JavaSourcesFiles is included in the same project directory as Product1/JavaSourcesFiles. If a Java project is created to work on Product 2, it would need its own copy of CommonFramework/JavaSourcesFiles. If CommonFramework/JavaSourcesFiles are instead placed in a separate Java project, there will be compilation ordering problems when there are mutual references between the common code and the product code.

Other examples could be constructed, but the above ones give the flavor of just how inflexible Eclipse is when dealing with source files arranged in perfectly reasonable ways.

Analysis

Note that although the complaints are coming from Java programmers using Java projects, it is not unreasonable to suppose that similar problems could arise when Eclipse is used for other kinds of development, such as C code or web pages. This suggests that the problem should not be construed narrowly, as just a problem just with JDT. The roots of the problem may lie deeper, in the Eclipse Platform itself; resolving the problem may require reforming both.

The most obvious restrictions are that all of a project's files must be contained under a single project root directory in the file system. Eclipse projects are single-rooted. Since a Java project's source and output folders are constrained to lie within the project, they thereby inherit constraints from the Eclipse project.

Even if the above restrictions were not the problem, there are the further restrictions that all files in the file system below the project root directory are automatically included in the project, and that all Java source files below a source folder are automatically included on the compile-time class path. In other words, Eclipse projects are all-inclusive, as are the source folders of Java projects. In Examples 2 and 3, the complaint is that there is no way to selectively exclude files.

Example 4 illustrates a third angle to the problem: that there is no easy way to share common Java source files between projects. In other words, Eclipse projects are non-overlapping. Nor, as Example 3 illustrates, is it possible to have overlapping source folders within a single Java project.

To summarize:

With respect to the files in the local file system, an Eclipse project is single-rooted, all-inclusive, and non-overlapping.
A Java project's source folders are all-inclusive and non-overlapping, and constrained to lie within a single Eclipse project.
A Java project's output folder is constrained to lie within a Eclipse project.

References

Eclipse bug reports touching on inflexible project structure:

Eclipse newsgroup discussions touching on inflexible project structure: