1. Background
2. The Need
3. What Is Invicta?
4. Invicta As A Framework
5. Technincal
6. Basic Usage Example
7. Future Development
The software development world in general and the J2EE development world specifically involve various complex build related tasks. This fact is not new. Makefiles processed by make have been a mechanism for transferring a project's source files into a final product for many years. However, some new software development standards of the recent years involve even more build aspects.
For example, a Java developer that wants to build and deploy a full Web Application on a J2EE application server has to do the following actions:
-
Compile the Java sources of his project's components according to the dependencies between the different components.
-
Create descriptor files for his Web or EJB modules.
-
Maybe pre-compile JSP files of this Web application.
-
Pack the compiled classes into JAR files, while taking care of dependencies between the JAR files.
-
Run an EJB compiler on the JAR files of the EJB modules for getting ready-for-deployment JAR files.
-
Pack the Web application into WAR files that include Servlet classes, descriptors and JAR files.
-
Pack all files of Java, Web and EJB modules into a single EAR file.
-
Deploy the application on a local or a remote J2EE Application Server.
As can be seen in the above example, a build process is relatively complex. It also varies from one project or developer to another (for example, no EJBs in project, automatic generation of code or descriptors, etc.).
The build process is usually performed by build scripts. Those scripts can be simple shell scripts, standard Makefiles, or XML files of the Apache ANT tool (mainly for J2EE projects). These tools are based on a project build process definition language. These languages are relatively complex and require some knowledge, especially when a strong build script is required.
The example above also shows that the actions required for building different projects are sometimes very similar to each other. This means that many software companies and developers spend time on writing almost the same build code multiple times, instead of sharing it.
Some graphical IDE (Integrated Development Environment) applications contain tools and wizards for compiling sources and building projects. These wizards ease the process of building a project. However, advanced developers cannot modify the default build process. Additionally, these features are IDE-specific, which means that performing automatic batch build processes, or sending the project to a developer that uses a different IDE or doesn't use one at all , is not possible.
So, if we have Make and ANT, as well as many others, why is there need for yet another build tool?
-
Because writing a build script is relatively complex, especially when you want to have a strong customizable build script.
-
Because there is no method for making everyone share the strongest script that someone once wrote. Too many people write similar build code: compilation, packing, deploying, etc.
-
Because the structure definition of a project is not handled. Large-scale projects that involve multiple components and dependencies are hard to manage.
-
Because there is no such concept as the definition of a 'class', for example a JAR 'class' that has the targets ('methods'): prepare, compile, pack, etc. Naturally, there is also no way to extend an existing 'class'.
-
Because each IDE has different concepts or wizards for defining project structure and build-related actions; no standard mechanism exists.
-
Because the project's structure, dependencies, component types, and build instructions are usually not stored as an integral part of the project sources (in versioning systems such as CVS).
-
Because there is no standard method for defining how a certain project depends on another existing project (of the same company or a different one).
From the above, it can be concluded what the requirements from a new build management tool would be:
-
Enable any developer to easily create a strong build script for complex projects without wasting time on learning a build script language.
-
Enable administrators to hide the complex build-script writing process, while preserving the ability to have a powerful build script per project, one which shares a one-time written build code.
-
Enforce a correct development methodology of dividing a development project into multiple components with well-defined dependencies and types of components.
-
Be very modular and easy to extend by administrators or developers. This modularity should let developers add their own extensions (mainly types of components).
-
Define a standard way for managing large-scale projects that involve inner and outer project relations and multiple components, while avoiding writing long dependency code and compilation classpath definitions.
-
Be a repository of strong and customizable 'build classes' that every company or developer can use instead of writing from scratch.
-
Do not depend on an IDE's specific wizard or definition tool. There should be integration with IDEs, but no dependence.
Invicta is a build management tool, targeted mainly for large-scale projects. Invicta does not replace existing build tools; instead, it uses them, functioning as a stronger and easier-to-use upper layer. This layer hides the complexity of the actual build script language and supplies additional functionalities.
While basing on Apache ANT, Invicta is not just an ANT extension; it should be generic enough to support in the future build tools other than ANT, and it is used by developers for processing their project's definition files and generating build scripts accordingly. Each developer is able to edit very simple XML files for defining his components, their types and their relations. These definition files become an integral part of the project, just like the project's sources.
Invicta addresses the needs of both 'simple' developers and administrator. For developers, the basic functionality of Invicta acts as a 'black box' for building a project, as shown in the following diagram:
For administrators or advanced developers, Invicta is a framework for their complete build environment. They can add their own customized types of components, which means writing once ANT code that can be used by all the developers in their project or company many times by simply specifying the new type name. Invicta's extensibility also allows accessing the information processed from the project definition files for writing output files other than ANT build.xml (for example, configuration files for a specific IDE).
In order to make Invicta a useful Open-Source project that developers can easily extend and maybe contribute to its development, Invicta must be a modular extensible framework. It should not be a 'closed' tool with a definite set of build functionalities. It should initially have only basic build functionalities, but should define standard methods for extending these basic functionalities. Such extensions can either be specific requirements of some developers or common functionalities that can be added to Invicta in an evolutionary way.
The following diagram shows the content of the Invicta project, the relations between its modules, the user-relevant parts and the extensible points.
The Core Engine is a Java application which processes Project Definition Files (components, dependencies, products, etc.), as well as Component Types definitions, required for the components of a specific project. The data collected can be accessed via an API it exposes. Output files (such as ANT build.xml files) are created by Dumpers accessing this API.
Component Types are XML definition files of build instructions for specific types (JAR, EJB, etc.). Each component defined in Project Definition Files specifies a type name that should have a matching Component Type. These are mainly templates of ANT code. For example, JAR-Component Type definition contains the code of ANT targets for compiling Java sources and packing them into a JAR file.
Each administrator or developer that is using Invicta is able to define additional Custom Types. These types can stand for themselves, or extend the built-in types for adding specific capabilities.
Template Handlers are how templates of Component Types access project or component-specific information. Essentially, they are Java classes that use the API of the Core Engine for accessing the information gathered from the Project Definition Files.
Dumpers use the API of the Core Engine for creating Output Files. As Invicta is modular, developers are able to define custom Dumpers in addition to the ones Invicta ships with:
ANT – generating ANT build scripts
Documentation – generating project documentation (Javadoc-like).
Project export definition – Generating a definition file of a project being distributed, to be used within a different project.
XML files of a simple structure. Each project has its own set of files that define its components, their types, dependencies between them and various other component and project settings. These files are the input for Invicta's Core Engine, which processes them and activates Dumpers on the collected information.
Output Files are Invicta's products. Dumpers create output files according to the information gathered from processing Project Definition Files. The main output file is ANT build script (build.xml).
As mentioned earlier, Invicta is not just a 'black box' taking definition files and transforming them into build instructions. Although many of its users will only use this aspect of the product, Invicta can do much more. As an open-source project, it is designed to be extensible by its users. This means that within the project, standard methods are defined for extending basic functionalities. By this, users are easily able to define the following additional modules:
-
Custom Types - additional specific Component Types (XML files with templates of ANT build script for building specific component types).
-
Custom Handlers - specific classes of Template Handlers for generating meaningful strings for build templates of Component Types.
-
Custom Dumpers - additional Dumpers that generate Output Files according to the information processed from a project definition.
Invicta is developed in Java. Its framework API is also a set of Java interfaces and classes. Some standard external packages are used by Invicta's code.
The Core Engine is a Java application that was tested using Sun JDK 1.3.1 and 1.4.1.
ANT's build script generated by Invicta was tested using Apache ANT 1.5.x.
Invicta was tested both on Red-Hat Linux and Microsoft Windows.
The Weta software company starts a new project called Solenopsis. The first version of the Solenpensis project contains four components:
-
lib – external libraries of Jakarta.
-
utils – a shared Java code of the Solenopsis project. Depends on one of lib's libraries.
-
core – the heart of the Solenopsis project. Contains Java sources and also some Weta proprietary files that should be processed during the build process. Depends on the utils component and uses a library of lib.
-
ui – a Web application component. Contains both Java servlets and JSP files. Depends on the core component. Should be packed as a WAR file ready to be deployed.
Weta's administrator installs Invicta and defines a new project. He also writes a new Custom Type named SolenopsisJAR that extends the basic JAR type for processing Weta proprietary files.
The following Project Definition File is written and maintained by Weta's developers:
|
<invictaProject>
</invictaProject> |
Weta's developers run Invicta on this definition file for generating an ANT build script.
Behind the scenes, the Core Engine will process the definition file, while checking its validity. It will also process the relevant Component Types. It will then activate the Dumpers. AntDumper will process the templates of the relevant types and will create an output build.xml file for the project.
Developers can now use ANT with the generated build.xml from command line or an IDE (Eclipse, for example). Some usage examples:
-
ant solenopsis.utils.compile - will compile the source of solenopsis.utils.
-
ant solenopsis.core.pack - will compile solenopsis.core sources and pack the classes into a JAR after building solenopsis.utils that solenopsis.core depends on. Weta proprietary files will also be processed.
Because the generated build script is strong and customizable, developers can alter the build process settings in run-time. For example, the following command should compile solenopsis.utils with debug information:
ant solenopsis.utils.compile –Dsolenopsis.utils.compile.debug=true
Invicta is far from being complete. There are already many suggestions and ideas for enhancement in the TODO list, among which are new Component Types, additional Dumpers and an Eclipse plug-in. Suggestions and RFEs are welcome.