David Boike NServiceBus Q&A

David Boike’s relationship with the NServiceBus goes back 10 years: He was building a system that ingested thousands of RSS feeds and wanted to find a way to scale what he could accomplish by multithreading on one server. Asynchronous messaging became the perfect way to scale out to multiple servers, as required by the load.

Today, David is a Solution Architect for Particular Software, the company that manages and markets NServiceBus, where his title can mean a lot of different things. At Particular, the focus is less on roles and more on goals: they operate through task forces formed to execute certain issues. One day he might be writing code, the next day writing documentation or creating a tutorial. “In a software product company, there’s a lot more to do than just code,” he says, “and I’m lucky to get to do a little bit of everything.”

David is also part of Particular’s Developer Education initiative, a core component of the company’s strategy. It’s sort of like a marketing force made up of actual engineers whose goal is to educate developers on how to design effective distributed systems through content creation, events, training (see Udi Dahn’s Advanced Distributed Systems Design course), webinars, and other initiatives.

Educating people on NServiceBus is a core part of our own team’s focus—see our Getting on Board with NServiceBus webinar with Chief Architect Jimmy Bogard. The questions we got at the end were thought-provoking, and we thought we’d tap David for his perspective on ones related to particular scenarios and use cases. He graciously offered his expert insight into your specific NServiceBus questions and shared his thoughts on the future of the tool, and microservices, generally. 

Q: What is your opinion on workflow solutions? I have trouble with the argument stating that pure event-driven architectures tend to get messy and you lose the overview.

The problem with most workflow solutions is you have to buy into the fallacy that complex business processes can be represented only using a graphical, draggy-droppy workflow tool. That works for simple cases, but as business requirements get more and more complex it breaks down pretty quickly.

Some tools will let you weave some code into that, but then you’re limited by that environment, so you’ve still got developers trying to code for all the edge cases in a tool that ties one of their hands behind their back.

Meanwhile, what becomes of our good software development practices when we use such a tool? What happens to good version control, code review, unit testing, continuous integration, etc.?

NServiceBus, on the other hand, is 100% code-centric and does not have visual tools to design a system. But we do have pretty powerful tools to visualize message flows through the system based on audit logs of what actually happened. These visualizations create a kind of living documentation of the system, provide visual proof that the system is operating as it was designed, and encourage more collaboration between developers and product owners.

We believe that transforming business requirements into working code is what developers are best at, and so our products are designed to make it as easy as possible for them to do that.

Q: How much information should an event contain? Is it bad practice to put whole records of data when broadcasting events to downstream systems?

The problem with stuffing events full of all sorts of data is the coupling that it creates. An event should be a mechanism for decoupling different parts of a system. It allows a statement to be made (something happened!) and then other parts of the system react in whatever way is appropriate—the publisher of the event neither knows nor cares.

When you have a “fat event,” all of a sudden there are as many reasons for that event contract to change as there are fields in the data. If any change needs to be made, then that change has to be reflected, both on the publisher and on any subscribers it has. Suddenly, things aren’t so decoupled.

Ideally, an event should make an incontrovertible statement that something happened (this is the event name) and contain only stable identifiers, and perhaps a timestamp to indicate when the information was effective, so that if messages arrive out of order you can tell which one is correct. For more information on this, you could check out our blog post Putting your events on a diet.

However, “thin” events are only effective if your system contains well-defined service boundaries, so that each part of the system is truly autonomous and doesn’t need to do request/response to other services to get information that was left out of the event. For more information on that, check out our webinar Finding your service boundaries: a practical guide.

One place fat events are just fine is inside a service boundary, where strong coupling is OK because you already have strong cohesion within the service.

Q: In the .NET Microservices book that came out last year, they were doing Domain Events (through MediatR) and Infrastructure Events (through Azure Service Bus, where you can add Higher Level abstraction for ease of use like NServiceBus). How do you feel about this separation of “events” ?

I think that aligns very well. Domain events are the “fat” events within a service boundary, and infrastructure events are the “thin” events exchanged across service boundaries.

Q: It seems like NServiceBus would be useful everywhere—this was very appealing.  What kind of system would this be overkill in, in your opinion?

We like to think NServiceBus is useful everywhere too. But all responsible developers should be acutely aware of the golden hammer fallacy.

If you’re creating a one-page app to take data from a user and shove it in one database table, just do that. You don’t need NServiceBus.

While you’re at it, don’t bother creating a business layer and a separate data layer and a separate API layer and all sorts of different DTOs and transformations to move data around between the layers either. Just take the data and put it in the database.

When that system evolves to the point where inserting data into the database sets off a process you need a flowchart to describe, that’s when you need NServiceBus.

Q: Can you explain how you differentiate and scale within NServiceBus to handle different requests with different priorities? In the right lane, messages can take as long as needed to exit and in the left lane messages need to be completed rapidly.

In NServiceBus a “logical endpoint” is the fundamental unit of scalability—essentially a named queue with a collection of message handlers to process messages that arrive on that queue. You can scale out an logical endpoint by deploying multiple instances of that endpoint all competing to process messages on the same queue.

Because NServiceBus uses type-based routing, you can use inheritance to create a “high-priority” version of the same command:

public class ProcessOrderForGoldCustomer : ProcessOrder {…}

One message handler processes both:

public class OrderProcessor : IHandleMessages<ProcessOrder> {…}

But because routing is handled by type, you can send orders for gold customers to a separate queue. Two logical endpoints share the same processing code. The high-priority handler endpoint can be deployed to 6 different servers, and configured to allow processing 24 messages concurrently. Meanwhile you configure the handler endpoint for regular orders only on one server, and only allow one or two concurrent messages.

In this setup, the high-priority handler can be scaled independently (either up or out) as needed in order to fulfill the SLA promised for the high-priority customers, while we really don’t care if the low-priority orders back up in the queue and take quite some time to complete.

Q: The NServiceBus webinar touched on how this async pattern changes the way we present UI feedback to a user; Could you talk a bit more about how you think about when to use synchronous vs. async handling, and how it changes your UI design thinking?

When you’re using asynchronous messaging, you definitely don’t want to have a grid (like the ASP.NET WebForms GridView) that you refresh from the server instantly after making a modification, as it’s possible the message hasn’t been processed on the back end yet.

Providing feedback to the user can be as simple as showing a dialog confirming the data was received, which gives the back end a second to process the message. In a healthy system there really shouldn’t be more than a few milliseconds delay before the message is processed. In other situations, you can use client-side script to “fake” the user into thinking the data was refreshed instantly.

In the most complex cases your app can have a pending tasks concept and receive updates via SignalR. I like to give the example of the Azure Portal. Some long running operations, like provisioning a virtual machine, can’t be completed instantly. So when you complete the wizard, a task is kicked off, and you find out when it completes via the Notifications area in the corner of the screen.

There are a lot of different strategies, and each works better in different situations.

Q: So, what’s new for NServiceBus—any updates or feature adds we should be looking out for?

Recently we’ve been focusing on tightening the integration and hosting story for .NET Core. We have a new hosting package that makes it much easier to host an NServiceBus endpoint in a .NET Core Generic Host application with a .UseNServiceBus(…) extension method. We’ve got another new package to integrate with the Microsoft dependency injection abstraction, so you can use whatever external container you want, as long as it supports that abstraction. That will eventually make all of our existing DI adapter packages obsolete.

We’ve also got enhancement releases on our “downstream” packages (packages to support a particular message transport, persistence, etc.) being released all the time. We’re on a mission to support native publish/subscribe wherever we can, so the days of having to add RegisterPublisher() calls in your routing configuration are numbered.

The best way to keep on top of our new releases is to monitor our discussion group’s announcements category, or the feed it generates. Does anyone still use RSS? You could always pull that into a Slack channel.

Q: Any personal thoughts on the growing popularity of microservices? What should people watch out for when jumping into it? What does this mean for the future of NServiceBus?

Microservices is a weird term because it means so many different things to different people. A lot of people seem to think it means a bunch of processes communicating over HTTP and deployed to Docker. But with that approach you end up with a ton of coupling and a distributed monolith. Those people would probably be better off with a normal monolith—at least it would be easier to debug.

My view is that microservices is just a clever rebranding of service-oriented architecture, and that all the same rules apply. NServiceBus fits nicely into this, but the key is careful analysis of service boundaries. Then inside a boundary, where you already have high cohesion in the data and business rules that the boundary contains, you can have a high level of coupling, and that’s OK. But between services, you use messaging to exchange only thin events, keeping coupling to a minimum. That’s how you achieve what microservices purport to deliver: the ability to evolve different parts of your system independently in a way that a monolith just can’t.

And if you’ve already got a monolith, you’re not really stuck. Check out Break that big ball of mud for some strategies to evolve toward a more microservice-like state.

Learn more from David and his Developer Education team on Particular Software’s Blog.

Contact an expert