This Week in Wonder

It's been a pretty busy week (and this is a lame effort to summarize 163 commit messages):

ERMovies
Ray Kiddy updated the movie example to 2008. New features include auto-populating DB and Apache Derby support.

Derby plugin
Used by BugTracker and ERMovies and now supported by ERPrototypes, it provides preliminary EOF support for Apache Derby. Most things seem to work, there are some issues with BLOBs, though.

ERXDelayedRequestHandler
If you ever have pages which *might* take a bit too long, but don't warrant a long response page, then this is for you. When you install this response handler (and use ERXApp) then a request that is taking too long is detached and the user is presented a "Uh, this might take a while, get some coffee" page - without you needing to do anything at all. And it has a "Stop now" button in case you don't actually want to finish the action.

This is also very useful when you are running in development and are dropped into a breakpoint. Normally, you'd get the adaptor error page but now it loops merrily until you continue.

loc:foo and negate:foo prefixes

These are the first implementations for the new prefix registration in the WOOgnl parser. They now also support carret bindings like ^title.

ERXLocalizerAssociation
loc:value = bar

first asks the current localizer to handle the value of bar. This is way more powerful and readable than before, where you would either need to use

value = session.localizer.bar

or write an accessor method.

ERXNegateAssociation
Ever had an accessor that was just the opposite of what you needed?

not:disabled = bar

helps here.

ERXRuntimeUtilities
There is no easy way in Java to stop one thread from another. Actually, it's not possible at all unless the thread is waiting in the exact moment you try to stop it. We added a few crucial points where the current thread checks it it should quit: whe fetching a row from the DB and when the context sets a current component.

You just call ERXRuntimeUtilities.checkThreadInterrupt() at regular intervals and the thread that wants to stop calls ERXRuntimeUtilities.addThreadInterrupt(Thread thread, String message).

ERRest
The REST framework is getting quite a bit of work done on it (with more coming soon).

The entity delegates resolver will now guess names that aren't registered (for instance, if you request /Manufacturer, it will look for a ManufacturerRestEntityDelegate), and you can now specify delegate names in Properties (ERXRest.Manufacturer.delegate=ERXUnsafeReadOnlyRestEntityDelegate).

The beginnings of the JSON input/output for ERXRest was committed, though not on by default (still experimental).

There are several new convenience methods for registering ERXRest. The simplest is to add:

ERXRestRequestHandler.register(new YourRestAuthenticationDelegate(), true, true);

to your Application constructor and add Properties entries for registering delegates.

ERXRest now supports non-integer keys. If a key in the URL isn't the property of an entity, it is treated as a key. You can now also specify a custom attribute on your EO's to act as the "id" value by setting:

ERXRest.Manufacturer.id=nameOfAttribute

, which allows your EO's to expose non-PK attributes as the lookup value. For instance, you could have a UUID "external ID" that can be used for syncing EO's across multiple applications.

Ajax Load-on-Demand
[MS: This is a repost from of the announcement the wonder-disc list]
This is the largest change to Ajax framework in a while, and because it introduces some breaking API changes, I wanted to explain what it's all about.

Previously, if you had a page that revealed a new Ajax component (new = first use on the page) only in an Ajax refresh, you are probably familiar with the problem that the necessary Javascript dependencies were not loaded correctly. The cause of this is that the component was hidden and did not have a chance to inject its dependencies into the head tag. With this most recent commit, Ajax framework (transparently) supports load-on-demand of Javascript dependencies. It keeps track of which scripts (and CSS) have been loaded on your page (both on the original render as well as subsequent Ajax requests) and if a new component introduced in an Ajax update requires a script that has not been loaded, it will switch to render the dependency with the new load-on-demand method.

The Javascript side of load-on-demand is actually a feature that appears that it will be in Prototype 2.0 which we have brought forward into wonder.js, and the Java side is just the usual Project Wonder black magic :)

So about breaking API changes and behavior changes:

Most of the ERXWOContext response rewriting methods have been moved to ERXResponseRewriter. They never really made a lot of sense on ERXWOContext, but especially now that these methods actually potentially consider information that crosses contexts, it made sense to reorganize them. Additionally, several of the API's have been changed -- most just to take a WOContext where they didn't before. The most notable API change is that previously the insertion behavior was controlled with two booleans which have now been replaced by an enumerated type that more clearly defines what the behavior of the insertion methods will be when the head tag is not present (either because it's just missing or because you're in an Ajax update). For most people, the only change you may need to make is to add the missing WOContext parameter if it's needed (the other variant of the method probably was not used very much).

As for behavioral changes, there is really one PRIMARY one (well aside from the whole load-on-demand thing). The big behavioral change is that previously you could actually "insert" script and css into the head tag BEFORE it actually existed. Generally this is only even an issue when you try to add script/css in your Page Wrapper by overriding appendToResponse and calling those method BEFORE super.appendToResponse (meaning your page wrapper has not yet rendered the head tag, but you're asking to insert into it). Because of some of the API changes, it was necessary to REMOVE this ability (long story, but the API's require a WOContext and the method that previously provided this ability did not have access to the WOContext to fulfill the API). The fix for this is relatively simple -- Just move any calls to those functions in your page wrapper AFTER the super.appendToResponse. This probably won't impact many people, but if it DOES impact you, it will manifest as your CSS and Javascript not injecting into the head tag and a warning will be generated on the console explaining what happened.

ERXStyleSheet and ERXJavaScript have been modified to register their loaded resources with the same system, so if you use an ERXStyleSheet or ERXJavaScript, it will not double-load if you later load scripts/css with ERXResponseRewriter. ERXStyleSheet will now SKIP rendering its stylesheet if it has already been loaded (so you can put ERXStyleSheet in Ajax rendered components in addition to using ERXResponseRewriter and it will behave the same). ERXJavaScript right now does not skip rendering, but it DOES register that it has loaded, so it will satisfy dependencies if you later use ERXResponseRewriter to load scripts on-demand.

If for some reason you want to turn off load-on-demand, you can 1) just keep doing things the way you were -- declare your scripts at the top level so they have all been loaded by the time the Ajax components render, or 2) set er.extensions.loadOnDemand=false.

While the actually lines of code of these changes is relatively small, it's conceptually sort of significant, so please try out your Ajax apps and let me know if you see any oddities. Hopefully this will make the components much more encapsulated and far easier to use for everyone ... Apologies for any backwards compatibility issues this introduces in the short term.

Misc Ajax
AjaxDroppable has an onBeforeDrop binding that lets you connect a javascript function before the drop happens. AjaxDroppable can also now act like a submit button instead of a link by setting submit = true. Both AjaxDraggable and AjaxDroppable got a shiny new paint job, too -- all the javascript that used to generate in the component output is now moved to wonder.js and the generated output is substantially smaller.

AjaxObserveField has a new onBeforeSubmit binding that allows you to return false to deny the submit.

Misc
ERD2WDirectAction now supports ErrorFoo?__message=bar calls which is an easy way to put up a error page.

We moved the work-in-progress stuff from ERXFetchSpecification to ERXGroupingFetchSpecification.

ERXObjectStoreCoordinatorSynchronizer: The synchronizer now exposes the ability for fine-grained sync controls between stacks. For instance, if you application is a write-mostly app, it may be undesirable to have inserts synced between EOF stacks. You can now setDefaultSettings on the synchronizer (or set the settings per-object store) to control exactly which types of operations will be synced across stacks, including turning syncing off entirely for particular stacks.

This Week in Wonder

This Week in Wonder's schedule is all thrown off, but to finally catch us back up:

ERXFetchSpecification
This is still new and under development, but ERXFetchSpecification extends the concepts of a fetch spec and adds:

• support for caching
• type safety via generics
• user info for tracking additional information as you pass a fetch spec around
• support for grouping results by keypaths

One of the primary features here is caching. In EOF, if you traverse a to-many relationship, the GIDs of the members of that relationship are cached, which ensures that you don't have to do a db roundtrip every time you want to traverse your relationship. Unfortunately, if you use a fetch spec, there is no such cache provided and you always hit the database. ERXFetchSpecification allows fetch spec results to be cached to avoid repeatedly hitting the database. Again, this is still being developed, but Anjo has some very cool ideas for integrating fetch caching with memcached.

WOOGNL
The WOOGNL template parser now supports registering custom association prefixes. You can now set WOOgnl.setAssociationClassForPrefix(associationClass, prefix). This allows you to implement associations like i18n:value="title", which could load the "title" key from your localized strings. We haven't yet provided any out-of-the-box association classes yet, but we have several ideas (happily stolen from other frameworks :) ) that we will be implementing.

JNDI/LDAP
ERXModelGroup has always allowed replacing connection dictionaries on a model based on Properties values, but it only supported JDBC adaptor models. JNDI models are now supported with the properties:


[modelName].serverUrl the per-model server URL to set
[modelName].user the per-model username to set
[modelName].password the per-model password to set
[modelName].authenticationModel the per-model authenticationMethod to set
JNDI.global.serverUrl the global JNDI serverUrl to use by default
JNDI.global.username the global JNDI username to use by default
JNDI.global.password the global JNDI password to use by default
JNDI.global.authenticationMethod the global JNDI authenticationMethod to use by default


Misc
The new ERXModelDoc component provides a convenient way to display model documentation for a model, entity, attribute, and relationship.

ERXStyleSheet is more versatile now and supports optionally inlining the stylesheet vs injecting into the head tag.

You can now provide a custom subclass of the WOOGNL template parser by setting the property "ognl.parserClassName" to your custom subclass.

Response compression now supports stream-based responses.

ERXInOrQualifier is no longer on by default. We had to remove it as the default in 5.4, anyway, because of compatibility problems. If you want to re-enable it in your application, add EOQualifierSQLGeneration.Support.setSupportForClass(new ERXInOrQualifierSupport(), EOOrQualifier._CLASS); to your Application constructor.

ERXQualifierTraversal provides an implementation of the visitor pattern on qualifiers.

ERXProxyAssociation allows you to wrap an existing WOAssociation and inject a fixed prefix or suffix to the existing value.

ERXExpiringCache now supports timeouts per-key rather than just a global timeout for all cache entries.

ERDLinkToEditObject can link to to-one's now.

ERXEnterpriseObjectChangeListener provides a base class for implementing cache invalidation or other classes that need to be notified of changes to specific entities.

JavaRESTAdaptor

I know ... There was no "This Week in Wonder" this week. Mostly because I had "This Week in Real Job". But here's a new framework in the Wonder family to tide you over till the Next Week in Wonder:

JavaRESTAdaptor
Note: My first post called this ERRESTAdaptor, but it's actually JavaRESTAdaptor, because all adaptors have to be named JavaXxxAdaptor. Whoops.

Overview
JavaRESTAdaptor is an EOF adaptor implementation on top of a RESTful web service, similar to ActiveResource in Rails. Currently JavaRESTAdaptor is read-only, but should be sufficient for consuming RESTful services.

Usage
The usage of JavaRESTAdaptor is best explained in an example. This example is adapted from an ActiveResource tutorial, but it provides several examples of common techniques. The service we're going to communicate with is a Beast installation. Beast is a simple Rails forum application. We'll use http://beast.caboo.se as our example service provider.

Creating the Model
Note that the instructions here assume that you're using the build of WOLips that builds tonight (to be able to pick "REST" from the adaptor list).

1. Create a Wonder project.


2. Create an EOModel (of any name) and choose "None" from the list of adaptors in the wizard.


3. In the default database config (assuming you're using a recent WOLips), select the prototype "EORESTPrototypes" and select the Adaptor "REST".


4. In the URL, enter "http://beast.caboo.se" and leave the username and password blank (currently JavaRESTAdaptor does not support HTTP authentication).

Creating Entites
Beast provides four entities that we want to interact with: Forum, Topic, Post, User.

1. Create an entity named Forum.


2. The "table name" of Forum will tell JavaRESTAdaptor the URL extensions and XML tags that can be used to access the entity. In the case of Forum, the XML tag name comes in two variants -- a singular and a plural form. For the Forum entity, set the table name to "forum,forums".
   
   Table name for JavaRESTAdaptor comes in several forms:
   • If you do not set a table name, JavaRESTAdaptor will simply use the entity name as the singular form, and use ERXLocalizer to generate a plural form.
   
   
   • If you only set a single value (i.e. "forum"), the adaptor will use that as the singular form and use ERXLocalizer to generate a plural form.
   
   
   • If you specify two values (i.e. "forum,forums"), the adaptor will use the first value as the singular form and the second value as the plural form.
   
   
   • Additionally, as you will see later, you can specify a set of possibly URL prefixes to use to access the entity. When you do not specify a URL prefix (as in our Forum example), the adaptor will assume that it can go to /[plural form].xml and /[plural form]/[id].xml to retrieve objects of this entity type.


3. We need to determine the attributes of Forum, so open http://beast.caboo.se/forums.xml in your browser and view the source. ActiveRecord (or more importantly Ruby) can one-up us here, because they automatically determine the attributes of records based on queries to the service. We ultimately want to generate Java code, so we need to create an EOModel with attributes.


4. Forum has the attributes: description, description-html, id, name, position, posts-count, and topics-count. Create attributes in your entity with the names converted to camelCase (description-html becomes descriptionHtml, etc). For each attribute, select a representative prototype, and set the external name to be the original name from the XML. For example, descriptionHtml is a "varcharLarge" prototype (we could pick one with a smaller restriction if we want to limit the values) and its external name is "description-html". postsCount is an "intNumber" and its external name is "posts-count". Do this for the rest of the attributes of Forum.


5. In Beast, forums contain topics, so let's create the Topic entity. Topics present an interesting issue with modeling, because Beast does not allow you to select topics from the top level, rather you can only select topics through a Forum. For example, there is no /topics/1.xml there is only /forums/1/topics/1.xml.
   
   This oddity does cause some problems for EOF, because EOF expects to be able to fetch objects with only a primary key. JavaRESTAdaptor provides the ability to model fetching these entities, but you should be careful of places where EOF may cause faults for individual objects. You may find it safer to model relationships to entities of this type in code with fetch specs, or if you're using the Wonder eogen templates, traverse these kind of relationships with forum.topics(null, true), which will force a refetch of the entities.


6. Request http://beast.caboo.se/forums/1/topics.xml to see an example of what topics look like. Follow the same procedure as for Forum, creating attributes from the XML. For dates, use the prototype "dateTime". For foreign keys, use the attribute "id" and mark them as non-class attributes.


7. Now to address the URL issue for Topics. Because there is no /topics.xml, we need to tell JavaRESTAdaptor how to find topic objects. Edit the Topic entity and set the table name to "/forums/[forumID]/topics,topic,topics". This says that to fetch a topic, you must use the URL "/forums/[forumID]/topics" where forumID is a variable that corresponds to the Topic attribute of the same name. Any time a fetch is performed on a Topic, a forumID must appear in the qualifier or an exception will be thrown from JavaRESTAdaptor. When traversing a to-many relationship from a Forum to a Topic, this happens automatically inside of EOF. However, if you are trying to fetch a particular topic, you must provide its forum in your qualifier to prevent an error.


8. Now create the Post entity. Posts are similar to Topics in that you cannot access them from the top level of the service, but they can be accessed in several different ways. It turns out that you can get posts for a user, for a forum, or for a topic (which requires a forum). JavaRESTAdaptor allows you to model these methods as alternative URLs in the table name by separating the URLs with a pipe. For instance, the table name for Post is "/users/[userID]/posts|/forums/[forumID]/posts|/forums/[forumID]/topics/[topicID]/posts,post,posts". I admit this is ugly, and ideally these definitions would be defined on the relationships instead of the entity, but unfortunately at the adaptor level, the EORelationship that was used to fetch the objects is long gone. What ERREST does in this case is that it looks at the qualifier provided and finds the URL that matches the most variables from your qualifier keys and uses that as the fetching URL. For instance, if your qualifier only has a userID in it, the /users/[userID]/posts URL will be used. However, if you qualifier has a forumID and a topicID in it, the /forums/[forumID]/topics/[topicID]/posts will be used. An exception will be thrown if a suitable URL cannot be found given the provided qualifier.


9. Lastly, create your User entity. Nothing fancy here, and top-level fetches are allowed, so take a look at http://beast.caboo.se/users.xml to create your attributes, and use the table name "user,users" on this entity.

Relationships
You have created all of the entities for reading information from Beast. You can now create the relationships between these entities. For entities that can be fetched from the top level, or when modeling relationships that will provide enough context for the fetch, relationships can be modeled just like a normal eomodel.

For instance, from post.user() is a completely normal to-one relationship because User can be fetched from the top level, and post will provide the "id" necessary to execute the fetch. Similarly, forum.topics() is a normal to-many relationship, because while Topic does not have a topic level fetch URL, traversing the relationship from Forum will provide the "forumID" necessary to complete the fetch.

The "problem child" relationships are, for instance, topic.posts(). Traversing the posts relationship on Topic will only provide a topicID. Unfortunately Posts requires a topicID AND a forumID. It turns out that this can be modeled by putting both topicID and forumID in the joins of the to-many relationship definition. The downside of this is that it will not be symmetric with the other side of the relationship, and will not be updated automatically when write ability is added to JavaRESTAdaptor. The other problem is that, while this technique works on to-many relationships, it does not work on to-one relationships. EOF does not allow a to-one relationship to be defined with a join on anything expect a primary key attribute.

In this situation, you will need to construct the relationship using a fetch spec, and not using normal relationship definitions in the model. Your fetch spec will work just like a normal fetch spec, but you must provide all variables necessary to complete a fetch. As an example, if you want to fetch a particular topic, you must fetch it like:


Topic singleTopic = Topic.fetchRequiredTopic(editingContext, Topic.FORUM.eq(aForum).and(ERXQ.equals("id", 633)));


or


aForum.topics(ERXQ.equals("id", 633), true)


Note that JavaRESTAdaptor allows fetching on non-class attributes.

Fetching Notes
When fetching against the remote service, only topic level EOKeyValueQualifiers, EOAndQualifiers, and EOOrQualifiers with a single entry will be processed for constructing the remote URL. Complex qualifiers can be constructed, but they will be evaluated in-memory against the most restrictive URL that could be constructed given your qualifier attributes.

Examples

EOEditingContext editingContext = ERXEC.newEditingContext();

NSArray forums = Forum.fetchAllForums(editingContext);
System.out.println("Application.Application: Fetching all forums");
for (Forum forum : forums) {
System.out.println("Application.Application: " + forum.name() + ", " + forum.postsCount() + ", " + forum.topicsCount());
}

System.out.println("Application.Application: Fetching forum w/ PK");
Forum singleForum = (Forum) EOUtilities.objectWithPrimaryKeyValue(editingContext, Forum.ENTITY_NAME, "3");
System.out.println("Application.Application: " + singleForum.name());

System.out.println("Application.Application: Fetching topics for " + singleForum.name());
NSArray topics = singleForum.topics();
for (Topic topic : topics) {
System.out.println("Application.Application: " + topic.title() + " created " + topic.createdAt());
}

System.out.println("Application.Application: Fetching posts for forum");
NSArray forumPosts = singleForum.posts();
for (Post post : forumPosts) {
System.out.println("Application.Application: " + post.createdAt());
}

System.out.println("Application.Application: Refetching single topic w/ PK");
Topic singleTopic = Topic.fetchRequiredTopic(editingContext, Topic.FORUM.eq(singleForum).and(ERXQ.equals("id", 633)));
System.out.println("Application.Application: " + singleTopic.title());

System.out.println("Application.Application: Fetching topic user");
User user = singleTopic.user();
System.out.println("Application.Application: " + user.displayName());

System.out.println("Application.Application: Fetching posts for topic (composite pk, which is kind of interesting)");
NSArray topicPosts = singleTopic.posts();
for (Post post : topicPosts) {
System.out.println("Application.Application: " + post.createdAt());
}

Post randomPost = topicPosts.lastObject();
System.out.println("Application.Application: Fetching the topic for a post (this will break if topic is not already fetched)");
System.out.println("Application.Application: " + randomPost.topic().title());

System.out.println("Application.Application: Fetch author of post");
User postedByUser = randomPost.user();
System.out.println("Application.Application: " + postedByUser.displayName());

System.out.println("Application.Application: Fetching posts by author");
NSArray userPosts = postedByUser.posts();
for (Post post : userPosts) {
System.out.println("Application.Application: " + post.createdAt());
}


Check out JavaRESTAdaptorExample project to see the model described here fully completed.

This Week In Wonder

The big one for this week is much better 5.4.x support. We've fixed quite a few outstanding issues and we should be in a pretty good place when the next 5.4.x release comes out (which is looking good).

Wonder/5.4 Changes
ERXWOForm
ERXWOForm works properly with the new 5.4 context.isSecure setting (defaulting to secure if context.isSecure())

ERXWOConditional/ERXElse
ERXWOConditional/ERXElse simply cannot play nicely with 5.4 wo:if/wo:else. In the 5.4 build of Wonder, a notable change is that you need to pick your conditional horse. In 5.4, WOConditional will no longer be automatically replaced by ERXWOConditional. WOOgnl's wo:if and wo:else will still resolve to the ERX variant, but if you use WOConditional directly and you want the Wonder behavior, you will need to search for all ":\s*WOConditional" and change them to ": ERXWOConditional". We looked at this quite a bit, but the semantic of the "else" implementation in 5.4 is very different than Wonder's, and continuing to replace WOConditional in the way that we do would break any 3rd party 5.4 components (like Apple's) that are written to expect if/else to work like 5.4's.

[edit: Oh -- I forgot to mention. If you DO NOT intend to use 5.4's if/else and you just want to switch to 5.4 without having to change your components, you can set er.extensions.WOConditional.patch=true and it will override the default disabled patching of WOConditional to be ERXWOConditionl. At the moment this should be OK, because none of the Apple components use their if/else, but in the longer term, you should switch over to ERXWOConditional if you intend to use ERXElse]

ERMemoryAdaptor
This .... actually builds now in 5.4.

ERXWOContext
Through a small bit of trickery, ERXWOContext URL rewriting fully works with 5.4's new context API's.

Foundation
The 5.3 Wonder NS* collections classes better match the 5.4 versions so there should be fewer compatibility issues (which was an issue if you were running 5.4 code with the ERExtensions project checked out).

FrontBasePlugIn and PostgresqlPlugIn
Both of these plugins have been updated/slightly mangled to properly generate SQL under both 5.3 and 5.4 (as long as you aren't calling the new 5.4 schema sync API's). Pierre has merged nearly all (except the latest feature where you can generate SQL without a running database) of the Wonder FrontBasePlugIn features into 5.4, and it fully implements both the 5.3 and 5.4 schema sync API's, so Apple's should be the preferred one under 5.4. Wonder still maintains the primary PG plugin, though, so you will be limited to 5.3 schema sync for now. If you're doing anything with schema sync/sql gen, you need to be very careful of what API's you are using -- it's a bit of a minefield in 5.4 when you're using anything except the built-in plugins.

Source
Almost all of Wonder runs under both 5.3 and 5.4 without mucking with your build path. However, because of some refactorings in WOOgnl, if you are checking out the Wonder source in Eclipse, you will have to remove the Sources_WO53 from your WOOgnl build path and add Sources_WO54 to your build path if you intend to use 5.4. There was unfortunately no obvious way around one.

Other Misc Things
Ajax/ERXFlickrBatchNavigation now support pagination of plain java arrays without using a display group, which should make it easier to paginate non-WOish collections.

ERXKey supports an easier API for filtering arrays. For instance:

NSArray filteredCompanies = Company.NAME.is("Mike").filter(arrayOfCompanies);

WOLips.framework and Click2Open now work under FireFox and IE.