Archive for category Uncategorized

gitignore Files… Are You Still Creating Them Manually?

Do you use Git? Do you create your .gitignore files manually?

If yes, here is an on-line tool that can make your life easier. Read the rest of this entry »

, , , , , , ,

Leave a comment

Alternatives for Limiting Message Size within an Event Driven Architecture

I’ve discussed using event messages to carry payload data, to help with resynchronizing a failed independent component with its loosely coupled neighbours. However, this could very easily lead to extremely large and inefficient messages, for what should ideally be very simple events. This will invariably lead to performance and scalability issues, not to mention affecting the cost of provisioning. Also, since message publishers are unconcerned with how many subscribers there might be, publishing large messages is an irresponsible development practice with potentially unforeseeable consequences. I’ve outlined some alternatives we’ve assessed in attempting to resolve this issue.

The first strategy is, as we’ve discussed, including a data payload within the event itself. For example, consider an order processor component raising an event to indicate successfully closing an order. The event might also include some customer data, which is not strictly necessary, just to indicate the order has been made. As I’ve indicated, this strategy is really only viable for small message sizes. If we were to attempt to include an entire order with the event, the overall payload size would dwarf the event notification and result in the issues we’ve already mentioned.

A second, leaner approach is to include a REST URL to reference the payload data. Thus subscribers can choose to consume the related payload data as required using the REST URL. In our example, a downstream payment processor component might need customer data from the order. By requesting the order customer details via the supplied REST URL, the payment processor potentially has access to the entire order. I like this pattern as it virtually eliminates the size associated with any payload and also leverages the caching benefits of REST, ensuring a flexible, efficient mechanism for very large payloads.

Note: Consider it a tip to include a version determining parameter in the URL, to ensure the reference within the event remains immutable in the face of any later changes to the data.

The third strategy is a little more difficult to explain and involves the use of a mediator object. A mediator is an abstraction used to lower direct coupling between interdependent components. The mediator is responsible for abstracting any communications between components and can be implemented in many different ways depending on its use and the data involved. Communications can be heavily cached, synchronous, asynchronous it really doesn’t matter. It is sufficient for the consuming component to know, the mediator is responsible for executing a required task which is ultimately under the purview of another component. A mediator may make a variety of communications but only ever to a single component, thus ownership of the mediator is clear.

This strategy is only to be considered where some behaviour is required, not just data, which is the responsibility of another component. Within our order example, it might be to obtain the customer’s billing address for the payment processor say. This could be considered behaviour because the customer’s billing address is unlikely to be a property of the newly closed order, and hence it would require a lookup based on the customer (a responsibility of the customer management component). The mediating object might orchestrate the process, acquiring data from the customer management component to do this.

Leave a comment

QCon London 2012 – Day 2

Just got back from another day at QCon London 2012. Previous day was very good, but this morning I wasn’t quite sure what to expect from the sessions, and surely it was a mixed bag. Originally, I’ve planned to see Architectures You’ve Always Wondered About, but since Martin Thompson’s introduction to Finance track was so good I thought to give it a go.

Key note for today – Simple Made Easy, by Rich Hickey – author of Clojure. Read the rest of this entry »

,

Leave a comment

QCon London 2012 – Day 1 of the Conference

Today was the first day of the conference, and it was a very good day. I must say it largely met my expectations. I was part of the day on Architecture track and part of the day on High Availability. But lets start at the beginning and the key note. Btw, no photos, my phone camera is useless :(

Read the rest of this entry »

Leave a comment

QCon London 2012 – High Expectations

I am in London this week for QCon London 2012. Expectations are high and I hope it lives up to it. So far so good, I have to say. I have been on two tutorials so far and if quality is kept at this level I should be in for a treat.

Read the rest of this entry »

Leave a comment

What’s Happened to Dynamic Data?

I came to TechEd 2009 with the intention of identifying what happened with Dynamic Data since last year. For those of you who don’t know; Dynamic Data is Microsoft’s answer to Ruby on Rails. It’s a means of quickly scaffolding your application with validated text boxes, date pickers and images based on the metadata included within your data source. So a database data type of VARCHAR(50) would automatically transform into a textbox with a size limitation of fifty characters say.
Read the rest of this entry »

, ,

Leave a comment

Predictive Reasoning

Devastatingly smart Rafal Lukawiecki, that virtuoso of technical presentation, gave yet another brilliant presentation on how the seemingly “ivory tower” techniques of Artificial Intelligence can be used in present day applications for unusual data validation scenarios.
Read the rest of this entry »

Leave a comment

Documenting Code

The most important thing I can stress here is that I truly believe in the value of good documentation and that this article only addresses what I believe is a flaw in some documentation strategies.

Usually a software document begins its life as an accurate description of what a system is doing and how it’s doing it. As the software evolves, the documentation doesn’t always evolve with it and sometimes in fact the divergence between what the software does and what the documentation says it does, becomes a problem. Now I know in theory that documents should be maintained but the reality is that this doesn’t always happen… In a perfect world software would be right first time and every time and requirements would never change, sadly this isn’t a perfect would.
Read the rest of this entry »

,

1 Comment

Indexing Nulls

Most database indexes types do not index NULL values. What this means is that anywhere a query’s WHERE clause contains “IS NULL” a full table scan will ensue (or at best an index range scan).
Read the rest of this entry »

, , ,

8 Comments

Scalability versus Efficiency

The tendency of a good developer is to make an application as efficient as possible. Minimise communications, use memory rather than disk for storage and generally make things as tight and self-contained as possible.

Efficiency is not scalability and developing an efficient application often runs counter to making a scalable application. Scalability is the ability to inexpensively grow to meet increased demand. It follows that, efficient applications do not necessarily support increased demand.

Let’s consider a sample efficient application. It confines a data processing task to a database that holds the data to be processed, in the form of some stored procedures. This approach has the benefit of reducing communication and shields the implementation of the process from the other tiers of the application.

Unfortunately, in enterprise applications, data processing applications are rarely simple. We all know that stored procedures are pretty bad as a programming medium (bad testing, debugging and IDE support, together with a nasty procedural syntax :-P). We also know that programs in a database can perform pretty poorly with all that context switching.

While efficient, such an approach will not scale well. A more scalable approach is to process the data away from the database and return the processed data back to the database. This is not efficient, requires configuration, and generally more trouble.
However, efficiency is not the purpose of this approach, scalability is. If each task can be broken down into discrete sub tasks, they can be split and parcelled out to many “cheaper” processors and hence your task becomes scalable. With the advent of Multi-Core technologies, this approach becomes even more attractive.
[tags]IT,Enterprise Applications,Scalability,3 tier[/tags]

Leave a comment

Follow

Get every new post delivered to your Inbox.