Saying LINQ is about databases is missing its true benefits
Just came across a
long discussion about
LINQ in Java on the ODBMS blog (thanks to
Miguel's tweet). There is some excellent discussion in there, but aside from a couple of people the discussion seemed to largely center on
- It's from MS so it's a bad idea to copy, and
- I don't think it adds anything over normal SQL syntax
The first is an unfortunate dismissal of a very powerful functional language construct because of its origin. And the second illustrates that the commenter does not truly understand what LINQ brings to the language in the first place.
Of course, I'd bet that 90% of .NET developers, if polled, would also equate LINQ with "type-safe SQL in the language", so this isn't a dig against Java people. Hopefully as Parallel LINQ gains some traction, this simplification will loose it's hold on people.
LINQ or Language INtegrated Query is really a functional way of expression operations on collections. And if you decompose a lot of code, anywhere where you are using loops to manipulating collections, LINQ is likely to create a more concise and powerful expression for the same operation. And being functional, the implementation of how LINQ does this is opaque to the caller. The caller simply describes what operations should be done on the data sources, allowing for optimization of the operations based on the data sources. That means they could be in a database, they could come from XML, or REST calls or simply exist as an in-memory object graph. But none of these things change the transformations desired.
I've only used LINQ once for SQL, although I use LINQ to objects, i.e. against IEnumerable<T>, almost daily and it's done away with a lot of foreach with temporary variables, temporary lists, etc. But even that scenario against the apparently now deprecated DLINQ or Linq2Sql illustrated how it wasn't just about replacing SQL with type-safe syntax, it allowed me to use one syntax for both database and local operations.
This project included doing a bunch of analytical processing against the data, including projection combining a number data sources. Not all of this was expressible in SQL and some of it wasn't a wise use of live DB queries, and performing the additional work in memory was a lot more efficient. Traditionally this type of work would exhibit a fairly obvious syntactic break between the local and the DB operations. And moving some part (say a sort of a sub-set) from SQL to local or vice versa would be a significant re-write. But using LINQ, the syntax was identical, it was merely a matter of deciding at what point the query should be turned into concrete data vs. a cursor against the DB. This is simply done by turning any IEnumerable to a List, forcing immediate execution of the query represented by the IEnumerable source. Either way, local or remote, the syntax stayed the same the power of where processing should happen was in my hands.
I do support the goal of getting something akin to LINQ in java, but I sure hope they don't attack the problem by creating some DB-centric query DSL. The greatest benefit of LINQ in .NET, imho, is that instead of hacking its syntax into the language, the building blocks of anonymous delegates, lambda expressions, anonymous types, var syntax and object initializers. Each one of these pieces is a fundamental part of C# 3.0 and can be used independently, but together they allow LINQ to exist. Discussion of common fluent APIs for databases or whole new languages like SBQL miss the benefit of "Language Integrated" in LINQ.
Going to be interesting to follow how this evolves, since I personally think that LINQ is one of the key differentiators between Java and C# that isn't just syntactic sugar (even though many think it is just that).
Labels: c#, java, linq
Dream access control
Just finished an
article over on the
MindTouch blog about tweaking Dream's default access patterns. I really like how Dream uses cookies, something you don't often see in REST services. Generally it's all about
X-My-Cool-Auth-Header business, which is yet another manual burden for developers. Not sure if this originated because people did raw http requests and either didn't know that most http request mechanisms have cookie support (even curl has a cookie jar), or whether it was a dislike of cookies.
The article also briefly touches on Prologues and Epilogues, a topic I need go into with more detail some time in the future. Basically every Feature call can have n pre and post actions that can do anything from checking authentication to mutating the request (think accepting data in json or Xml and having a prologue and epilogue do transformations on the way in and out so that the feature itself doesn't have to worry about the data format but can assume that it always gets Xml. The system kind of reminds me of apache handler chaining from mod_perl.
Labels: c#, dream, mindtouch, rest
High Scalability article about notify.me
Today,
High Scalability ran an
interview with myself about the architecture behind notify.me. The interview covers a general overview of what I will cover in more detail over on the
notify.me tech blog.
Labels: async programming, c#, notify.me
Dream for REST and async
I've been doing a lot of work inside of
MindTouch Dream as of late over at
MindTouch and i'm really digging it. Steve's put together an awesome framework for doing asynchronous programming on .NET and for being able to treat all access as RESTful resources in your server side code.
Now, coming from a very Dependency Injection heavy design philosophy, Dream has been a bit of an adjustment for me, but the capabilities of Dream, especially the coroutine approach for dealing with requests, is very powerful and fairly intuitive, once you get your head around it.
In an effort to ease the Dream learning curve and cement my understanding of the code base, I'll be blogging articles about it as I go along, and cross-posting them to the MindTouch developer wiki as well. My first article was a continuation of Steve's Asynchronicity library series, this one about coroutines (read: yield) in Dream.
I've been using the C# Web Server project for my REST work up until recently, but I'm currently in the process of migrating it over to Dream. It just removes all the legwork and fits much better into the async workflow of the rest of notify.me.
Clearly I am biased, but seriously, if you need to build REST interfaces in .NET, Dream beats anything you can roll on your own in a reasonable amount of time, and definitely is about 1000% more powerful than trying to force WCF down the REST or even POX path.
Labels: async programming, c#, dream, mindtouch, rest
C# 3.0 language features vs. Java's OS community
I've been in a constant struggle for a number of years and it's a battle between wanting to program in C# but envying the variety of open source projects available on the JVM.
And this isn't so much a windows/linux thing, because mono vs. native .NET is really the least of my problems. The mono team has done an amazing job letting me write code without worry in Visual Studio and deploy on linux. With mono 2.0 just out, i have no code left that doesn't just work when i copy the dll's over. The fact that i do a lot of .NET on linux is not a pain point for me.
But when it comes to open source libraries, the eco-system on java is just so much richer. Think of an API or protocol and google it for java and you'll find it. Do the same for .NET and your chances are much smaller. And if you find it, chances are it's a port of Jfoo to Nfoo. So you could say, I've got library envy.
So why not just go java? Well, there are a number of C# language features that many might accuse of being "sugar", but i find them incredibly useful to express my intent in code. Here are the things that keep me in C# 3.0:
- Delegates, especially anonymous delegates. For concurrent programming, they are essential
- Events. Sure, they are just more sugar on top of delegates, but for async, having the ability to easily declare, subscribe and fire events are a god sent.
- Properties. Public fields can't be part of an interface and if you ever need to attach observation to the value change, then fields are a dead end. And seriously, getFoo()/setFoo() is busy work for the compiler. It's less readable and more error prone.
- Lambda expressions. Yes, i want a taste of functional with my imperative
- LINQ. And i don't mean LINQ to SQL. This isn't about databases, this is dealing with collections, doing map, reduce, filter and projection. Yeah, you can loop by hand, but it's so much less expressive or readable.
- var. Type inference is the way to go for static type languages. Less clutter, still strongly typed. You can get most of the benefits of dynamic languages with type inference and not have to give up type safety. I hope that we get closer to duck typing tho.
- Dead easy interop with native processes. No, my VM is not some abstract world that shan't know about its environment. It's just a better way to code, so let me do machine specific things without making me pay penance in blood for it.
- A language with much more innovation. Ugh, I know MS made innovation a dirty word in their anti-trust battles, but it's really true for C#. C#2.0's best features were aped by Java 5 and C#3.0 is once again miles ahead in new capabilities. People might argue whether that's a good thing, but it can't be disputed that C# is innovating much more rapidly.
So there you go, I'm a C# fan boy. I admit it. But still, i got this library envy. What's a guy to do? I see two possible paths, C# on the JVM or java libs on mono. Both are mostly possible, but I really need to do some spikes to know where the pain starts with the two options.
C# on the JVM: Mainsoft Grasshopper
Author in C#, cross-compile IL into java byte code. Mainsoft is apparently collaborating with the mono guys, so generally what makes it into mono in terms of language features will make it into
Grasshopper. Need to take some of my projects and see what happens when i try this.
Java on C#: IKVM
Compile java code into .NET IL using
IKVM. This is dependent on the state and compatibility of IKVM and Java. Need to take some popular java open source projects and see what happens when i try to build them using IKVM.
Either sounds like inviting a life of debugging edge cases that no one else cares about. But I don't see a better option. Am I overlooking some other path? Should i stop whining and just go java? Is the OS situation not nearly as bad on .NET as i make it out to be and I can just continue as is? Things that keep me up at night.
Labels: c#, grasshopper, ikvm, java
Synchronicity Kills
Finally got around to starting my blogging about the
technology being built for
notify.me.
It's pretty funny that both my
Mindtouch and my
notify.me coding revolve around heterogenous, asynchronous programming to achieve highly scalable concurrency. Since i started working on notify.me before I started at Mindtouch and dug deep into
Dream, the approaches are currently rather different, although I think i can bring lessons learned from either to the other as things progress.
IEnumerable.ForEach()
I keep wanting to do ForEach() on a collection, and noticed that it was inconsistent whether that extension method was available or not. I always figured that i wasn't importing the right namespace, and blamed ReSharper for not being smart enough to figure out what namespace I was missing. So I finally googled the problem only to find
this. Turns out ForEach() isn't on IEnumerable
, only on Array and List. Meh. But thanks to Dave, i now have it in my core assembly.Labels: .net, .NET 3.5, c#, Extension method
A case for TDD
I know, it's been rather quiet here lately. I've just been slammed with coding, so writing things up is falling behind. In addition, my blogging time's going to be split between
techblog.notify.me and
www.mindtouch.com/blog. And I'm behind on both of those as well. I should have some fun
Dream and
DekiScript stuff for the mindtouch blog and some asynchronous programming for the
notify.me blog soon. As soon as i can get myself to stop coding again.
So what makes me stop coding for a minute to babble on? It's just a quick case studio of why TDD is important.
I've been an Inversion of Control/Dependency Injection for about a year and a half, and while I've eased my way into it, I'm pretty much at the "an interface for every class" stage of having everything abstracted so i can easily mock things. But here and there, I take in third party assemblies for my projects. And most of the time, they are not well interfaced. And generally I try to create a facade that is interfaced, so i can test my interaction in isolation. But depending on how many secondary classes their code uses, sometimes my facade gets lazy leaving places i can't mock.
Now, i'm pretty religious about test coverage, but i do have holes where my facade leaves untestable bits. And this is where TDD shows it's worth. Because when a feature is added or a refactor happens, almost with 100% certainty, the bugs that manage to get into production are in the code that doesn't have test coverage.
The lesson here is that the time saved in not building a properly mockable facade, thereby torpedoing my testability, is repaid manyfold in debugging later as bugs make it into production. meh.
Labels: c#, mindtouch, notify.me, resharper, tdd
