Module | Status | Priority | Test Coverage | Progress | Who |
---|---|---|---|---|---|
1. Document Storage | |||||
1.1. Collection storage | Stable, but subject to redesign | High | No tests | 0% | wolf |
The current organization of collections and resources causes a number of problems with respect to (a) locking, (b) query performance, (c) update performance. Right now, documents are tightly bound to the collection in which they are contained. Any operation on a document has to go through its parent collection. As a result, locking and access control becomes quite complex as we need to take care of the document and the collection. There's also a direct dependency between the size of a collection (in terms of the number of documents stored in it) and document update speed. If a collection has a large number of documents, removing a single document becomes very slow. This problem can be solved by physically decoupling documents and collections:
| |||||
1.2. Allow metadata to be associated with a document | Open | Avg | N/A | 0% | |
Metadata could include system properties like last-modification date or user-defined metadata. Preferably, metadata records should be ordinary XML documents. The format should not be restricted. | |||||
2. Indexing | |||||
Since version 1.2, there are now alternative index configuration methods which support the optimizer in rewriting a query for best performance. A new modularized indexing architecture allows to plug in arbitrary new indexes into the indexing pipeline. A N-gram and a spatial index module were added as prototypes to test the new architecture. There will be other index types added in the future, for example:
| |||||
2.1. XQuery and XPath Full Text 1.0 | Open | Avg | N/A | 15% | |
Implement the W3C's full text extensions to XQuery. Many features could be easily implemented on top of Lucene. The grammar for the full text extensions has already been merged into the XQuery parser as part of a Google Summer of Code project and is available in an SVN branch. It just needs to be filled with life. | |||||
2.2. Index-support for order-by, distinct-values | Open | Avg | N/A | 0% | |
Order-by expressions and other functions that need to access atomized nodes are not supported by indexes. | |||||
2.3. Collation-driven indexing | Open | Avg | N/A | 0% | |
Maybe part of FT index redesign. | |||||
3. Transactions and Recovery | |||||
The journal log and the recovery manager should be stable and are covered by extensive tests. However, recovery failures can not be excluded entirely. The tests can't reproduce every possible real-world scenario. However, some steps remain for eXist to become a fully transactional database system. Transaction support is currently limited to the functionality needed for crash recovery. Though we maintain transactions internally, they are currently not exposed to applications. Also, read operations are not transactional right now. In order to allow user-defined ACID transactions with support for rollback, all index files would need to be protected by the journaling log. The required functionality is basically available, but the feature is currently not regarded as high-priority. | |||||
3.1. Internal transaction management | Stable | x | Tested | 100% | |
Transactions are maintained internally, but they are not exposed to applications. eXist-db does not yet support full ACID transactions. Read-only operations bypass the transaction system. | |||||
3.2. User-definable transactions | Open | Low | N/A | 0% | |
Journal logs are limited to critical data required for recovery. No transaction rollbacks. | |||||
4. Backup / Restore | |||||
4.1. Store configuration into backup | Open | x | No tests | 0% | |
The backup utility should also include a copy of the relevant server configuration files (mainly conf.xml) into the created backup. Settings like page size, additional XQuery or index modules etc. are important. | |||||
5. Configuration | |||||
5.1. Dynamic configuration of the database via Java Management Extensions (JMX) | Open | x | No tests | 0% | |
Main problem: access control and security. | |||||
6. Node-level updates | |||||
6.1. XUpdate | Stable | x | Tested | 100% | |
No remarks available. | |||||
6.2. XQuery Update Extensions | Stable, but subject to redesign | x | Tested | 75% | |
eXist's XQuery Update Extensions were implemented before the first W3C drafts were published. The differences between the W3C recommendation and our implementation are not that big, though there are some subtle differences concerning the processing model. In fact, the recommendation simplifies a few issues. | |||||
7. Access-Control | |||||
The currently implemented Unix-like access control scheme is sufficient to protect resources and collections in a multi-user environment. However, it might be too coarse-grained for some types of applications. A more dynamic ACL implementation could help here. Right now, security management forms part of the database core. This is unnecessary. A more modular architecture would allow different security managers to be plugged in. It would be the responsibility of the security manager implementation to handle ACL lists. Since version 1.1, eXist supports the XACML standard for fine-grained access control to stored XQueries, Java classes etc. | |||||
8. Schema Validation | |||||
8.1. XQuery validation features | Open | Avg | N/A | 0% | |
No remarks available. | |||||
8.2. Store PSVI with the node tree in the database | Open | Low | N/A | 0% | dizzzz |
No remarks available. | |||||
8.3. Static typing based on PSVI | Open | Low | N/A | 0% | dizzzz |
No remarks available. | |||||
9. XQuery | |||||
The XQuery engine as well as the standard function libraries should be updated to align with the XQuery 3.0 recommendation. Basically, almost all XQuery 1.0 core language features are implemented, excluding schema related features, which are currently beyond eXist-db's scope. XQuery support in eXist-db is covered by the official W3C XQuery Test Suite (XQTS) 1.02. Implementing the official XQTS XQuery test suite was a top priority in order to guarantee standard conformance and avoid future regressions. | |||||
9.1. Core XPath and XQuery | Stable | x | tested | 100% | |
Updated to the XPath 2.0 and XQuery 1.0 recommendations. Stable, excluding schema-related features. | |||||
9.2. XPath and XQuery atomic value types | Stable | Avg | tested | 99.4% | |
9.3. XPath and XQuery function libraries | Stable | High | tested | 99.4% | |
Updated to XPath 2.0 and XQuery 1.0 recommendations. Stable, excluding schema-related features. | |||||
9.4. XPath and XQuery function libraries | Stable | High | tested | 99.4% | |
Updated to XPath 2.0 and XQuery 1.0 recommendations. | |||||
9.5. XQuery serialization | Stable, but subject to redesign | Avg | tested | 80% | |
Though we implement most of the serialization options specified in the XQuery and XSLT serialization spec, some options need to be reworked and should be covered by tests. | |||||
9.6. Error reporting | Stable, but subject to redesign | Avg | N/A | 75% | |
Error reports by the XQuery parser and compiler need to be improved. | |||||
10. XInclude | Stable | Low | No tests | 80% | |
XInclude expansion happens at serialization time. Queries across the included document fragments are not possible. Stable, but limited. | |||||
11. Interfaces | |||||
11.1. XML:DB API | Stable | x | Tested | 100% | |
No remarks available. | |||||
11.2. XML-RPC | Stable | x | Partially tested | 100% | |
Exposes the entire database functionality. | |||||
11.3. REST | Stable, but subject to redesign | Low | Partially tested | 90% | delirium |
Does not cover administrative functions, e.g. user-management and permissions. Stable, but further functionality could be exposed. The XQuery API for Java (XQJ) implementation makes heavy use of REST and improves the interface in some aspects. However, the XQJ branch has not yet been merged into trunk. | |||||
11.4. SOAP | Stable | Low | No tests | 90% | |
12. Documentation | |||||
12.1. 1.5 | Open | Low | N/A | 0% | |
Based on Java 6. | |||||
13. Other Tasks | |||||
13.1. i18n | Open | Low | N/A | 20% | |
Provide translations for error messages, console outputs etc. At least, resource bundles should be used, so others can translate them if they want. | |||||
13.2. Clean up/upgrade libraries | Beta | Low | N/A | 60% | |
All libraries included with eXist need to be checked. | |||||
13.3. Move to ANTLR3 parser | Instable | High | N/A | 60% | ljo |
Change the parser to ANTLR3 which is better performant, LL* lookahead capable and processes whitespace in a better manner than ANTLR2 which we currently are using. Maybe use gunit for testing? |
Percentage | Description |
---|---|
0 | work not started |
20 | 1-20 Percentage of completion |
40 | 21-40 Percentage of completion |
60 | 41-60 Percentage of completion |
80 | 61-80 Percentage of completion |
99 | 81-99 Percentage of completion |
Done | 100 Percentage of completion |
Priority | Description |
---|---|
1. Highest | |
2. High | |
3. Avg | |
4. Low | |
5. x |