Monday, January 31, 2005
DIY Serialization
Forgot about putting this up as promised. Of course, this is a very simplistic example with only one variable sized field. If you had more, you'd have to also serialize those field's sizes and reconstruction becomes a bit more complicated. But the important part about this, is really the whole concept of pinning some memory down so that you can manipulate it directly and then releasing it back to the control of the Garbage collector. Pretty cool, really.[StructLayout(LayoutKind.Sequential)] struct DIYSerialize { public Int32 Id; public byte[] Data; public DIYSerialize(Int32 id, string data) { this.Id = id; this.Data = Encoding.ASCII.GetBytes(data); } public DIYSerialize(byte[] Raw) { Int32 size = Raw.Length; IntPtr pnt = Marshal.AllocHGlobal(size); GCHandle pin = GCHandle.Alloc(pnt,GCHandleType.Pinned); Marshal.Copy(Raw,0,pnt,size); this = (DIYSerialize)Marshal.PtrToStructure(pnt,typeof(DIYSerialize)); pin.Free(); } public byte[] Serialize() { Int32 size = Marshal.SizeOf(Id.GetType()) + Data.Length; IntPtr pnt = Marshal.AllocHGlobal(size); GCHandle pin = GCHandle.Alloc(pnt); Marshal.StructureToPtr(this,pnt,false); byte[] d = new byte[size]; Marshal.Copy(pnt,d,0,Data.Length); pin.Free(); Marshal.FreeHGlobal(pnt); return d; } }
Saturday, January 29, 2005
Interop vs. Serialize
I was this close to just using ISerializable for by binary representation for networking. But then talking to n8, i decided that leaving the door open for java interop was important enough in Enterprise computing, that I couldn't ignore it.
So, I'm back to coming up with my packet structure in manner that i can easily serialize by hand. Right now the plan is something along the line of a fixed sized header followed by a variable sized gzip-Xml chunk and a variable sized raw data chunk. Both of the variable chunks can be 0 length. The header is used for expressing the purpose of the packet, and describing the offsets of the variable packets. The gzip-Xml is for data that the recipient is supposed to parse and act on, while the raw data packet will be just that, raw data that i sued to construct a complete file.
I need to dig up the test code i did for doing manual serialization in .NET. Really, just what would be done in C with a struct and memcpy, but of course, since in C# you don't have pointers and the memory is managed for you, it's a bit more of a hoop jumping exercise. The code is on another machine, so i'll leave that for another post.
Monday, January 24, 2005
More on Mac Mini Media Center
Some good discussions and links can be found here and here. And a USB to TOSLink Digitial Audio option.Friday, January 21, 2005
Fun with enum bitflags
I've previously learned about the enum [Flags()] attribute, but had forgotten usage and reference to it isn't the best, so i figured, iput this down before it slips my mind again.
Say you want a compact way to store a bunch of boolean flags but you want it to end up as human readable code and you don't want to deal with binary operations. Step 1) use an enum for your flags and give the enum the [Flags()] attribute. Now you have a human readable flags, but you still have to do bitwise operations on them to set, unset and get these flags. So, wrap the whole thing in a struct giving a boolean property as accessor for each flag.
struct ByteFlag { // Internal storage of our flags ----------------------------- [Flags()] enum flags : byte { IsCool = 1, IsTasty = 2 } flags bf; // Initialize, set and retrieve the underlying storage ------ public byter(byte pInit) { bf = (byteFlag)pInit; } public byte ToByte() { return (byte)bf; } public void FromByte(byte pValue) { bf = (byteFlag)pValue; } // Accessors ------------------------------------------------ public bool IsCool { get { return (bf&byteFlag.IsCool)==byteFlag.IsCool; } set { bf = (value)?bf|byteFlag.IsCool:bf&~byteFlag.IsCool; } } public bool IsTasty { get { return (bf&byteFlag.IsTasty)==byteFlag.IsTasty; } set { bf = (value)?bf|byteFlag.IsTasty:bf&~byteFlag.IsTasty; } } }
Now you can do store that flag in a byte and when you want to manipulate the flags it's as simple as:
ByteFlag byteFlag = new ByteFlag(theByteOfStorage); byteFlag.IsCool = true; Console.WriteLine("IsTasty: {0}",byteFlag.IsTasty);
Thursday, January 20, 2005
Could you please sign this?
Right, so i started thinking I'm doing too many trips, because the client and server could just sign their packages. But that doesn't actually prove anything. After all, someone could have just connected to a server, received their package and then used it to respond to someone connecting to them. That way they'd be pretending to be the other server. The package would be validly signed but not prove the servers identity. Basically we do still need the random phrase challenges from both sides
This does bring up the whole Man-In-The-Middle problem. I could still simply become a proxy between two peers. Not wanting any central service for registering, I'll probably just go with adding the IP in both the client and server packets. That way we can check that the message originates from the same IP we're talking to.
RSA PK encryption with .NET and Mono
Experimented with System.Security.Cryptography.RSACryptoServiceProvider last night to see if i could use it for Public Key encryption between .NET and Mono. Worked flawlessly.
This goes back to my networking thread. I'm currently contemplating using PKI to establish trust relationships between peers. For the sake of this discussion, there is a serving peer, the one that is connected to, hereafter the server, and a client peer, the one that makes the connection, hereafter the client.
Each peer keeps the public keys of its trusted peers. When a client connects it sends its own public key to identify itself. The server first checks whether it trusts that public key at all, then, having passed that test, encrypts a packed containing a random string, plus its own public key with the client's public key. Only the true owner of the public key can decipher the message.
Now it's the client's turn to check whether it in turn trusts the servers public key. If it trusts the public key, the client encrypts a new packet containing the original random string, plus one of its own divising with the server's public key and sends this back. Again, only the true owner of the public key can decipher the response to its own challenge.
Decrypting the package the server can now verify that the client is truly the owner of the public key, and finally it sends the client's random string back encrypted with the client's public key as acknowledgment of its trust and affirming its own identity to the client.
At this point the client should send all the public keys it trusts to the server and the server responds with the subset it trusts as well . Now we have a relationship that allows the peers to trust each other and know what information they may pass on directly, given the others level of trust and what information it would have to proxy, being the trusted intermediary.
I don't know if I'm re-inventing something obvious here. It seems fairly similar to the SSL handshake, except that I don't care about encrypted communication, and therefore don't use SSL. I just want to establish trust on connect.
Wednesday, January 19, 2005
MacMini, the next set-top box?
Ok, not quite programming, but close enough.
I was looking at the MacMini and were I not only invested in using my chipped Xbox with XMBC, I'd seriously consider a MacMini as my Media Center. I mean it's perfect:
- Has DVI out, easily connectable to my HDTV for high res output
- Has a DVD player and a DVD drive
- Is tiny
- Is ultra quiet
- Runs iTunes, and all things XMBC considered, iTunes is still the best music player
- Has BlueTooth for all sorts of remote control options, from Mouse/Keyboard to PamOS to Phone
- Easily customized via Applescript or perl or even just writing an app in C++, Java, C# (Mono)
- Easily networked
- Affordable
EnvDTE going nowhere
Ok, no answers on my quest on what I can cast Events to. No matter, I can always add better support for Events later. Time i get back to it and start adding "add code" drop downs to the code browser. We'll see how it goes from thereFriday, January 14, 2005
Gzip under .NET
The kind folks over at ICSharpCode have created a Zlib library with Gzip, Zip, Tar and Bzip2 implementations.
Tried it out last night and it works beautifully. It fits into the networking thread of last in that I've decided that for communication i'm going to use Gzipped Base64 encoded XML. That gives me a nice compact format that's quite portable. Already tried out sending a such encoded message from a perl program on Linux to .NET on Windows and it worked beautifully. And it's 90% smaller than plain old XML.
Maybe if the Fast Infoset Project takes off, there'll be a more compact binary XML. In the meantime, this approach is at least easy for anyone to read, what with the ubiquity of gzip and b64.
Wednesday, January 12, 2005
Remoting and Mono
After my TCP toying I decided to do some Remoting experiments last night. One of my goals with the network programming i'm doing right now is that the business logic of the code should be portable between .NET and Mono. I know Remoting is a probably not the answer there, since it may not even stay compatible between .NET revisions.
Regardless, I tried it out and it worked great between Linux and Windows. Nice, simple elegant (with a fair share of black magic under the hood). The only pitfall I came across was that if you are using RemotingConfiguration.RegisterActivatedClientType(), you gotta make sure that the type you are using comes from the same assembly on the two platforms. I had a bunch of problems because I compiled the Windows and Linux versions differently at first, but then i just copied the Assembly that contained the classes to be remoted to the linux side and compiled against it. That did the trick.
Now, is there something inbetween Remoting and writing my own protocol with TcpClient for doing Client/Server communication? More research for another night.
Monday, January 10, 2005
TCP communication
Played around with TcpListener and TcpClient a bit this weekend. Last time I did low level network programming was in perl and it's interface was basically a socket interface. I did multiplexing with Posix select to check on sockets that were ready to do some communication. With C# i decided to go the multi-threading route instead and use the slightly higher level interfaces TcpListener and TcpClient.
I use the listener in the main thread in non-blocking mode, using pending to decide whether to accept a new connection. Then i spawn a thread for each TcpClient and have it listen the the client.
So far so good and fairly simple stuff. Except that I can't figure out how to detect if a client closed the connection on me rather me disconnecting them. I thought i'd get a SocketException when the client goes away, but so far my experiments show TcpClient happuily trying to read from the socket forever. Ho hum.
Update: Ok, so i'm just blind. I was so wrapped up looking for exceptions telling me that the client went away that i ignored the normal method of checking whether the read call read 0 bytes, just like in normal Posix sockets. Now that works like a charm
Friday, January 07, 2005
Code Formatting
Found a cool tool for formatting C# code for HTML. Needs some tweaking to work right in this template tho...
EndDTE vs. VCCodeModel
Trying to identify all the members in a class, i can use CodeElement.Kind and build a large case statement against vsCMCodeElement.. But once I hit vsCMElement.vsCMElementEvent I hit a dead end. All the others have CodeBlah interfaces I can use to get .Access from, but there is no CodeEvent.
Digging around the net for a while i found the Microsoft.VisualStudio.VCCodeModel namespace, which has VCCodeEvent which in turn doesn't have an .Access member. And anyhow how does this sucker relate?
It would be nice if there was a reference from the vsCMCodeElement enum to the appropriate CodeElement interface.
And while we're wishing, how about a C# to EnvDTE reference. Oh, well, more experimenting to come
Thursday, January 06, 2005
Designing classes
Lately I've been trying to do a lot more design before sitting down and cranking out code, thinking that it would cut down on some of the iterative revisioning I see being the staple of programming. But how to do that design without having to do it all over again once the coding starts.
UML is cool and all, but I find it better for explaining things to others, after the design has progressed, than for designing objects. It also doesn't let me express C# syntax properly and I still have to transcribe everything by hand into the IDE later on and the link between the two is lost.
I could write out the object design as text, describing inheritance, members, etc. That usually works best for me, but then I still have got to transcribe it into the IDE and again the link between design and implementation is lost.
I briefly played with code generation, creating an XML schema to define the classes, then generating code from that. Still suffers from the design/implementation disconnect. As soon as the generation is done, and i fill in the code, they diverge. I considered writing the Code Generator to be able to parse the code back in, update the XML and then be able to generate any changed code from that.
I think I've stumbled across a better approach though. Integrating straight into VS.NET and using the CodeModel that the IDE builds on the source files, to alter and design classes. Everything stays in the native code format, but i can still tweak it without touching code.
My current design goal is an Add-in that mimics the Object Browser, but allows you to alter the classes on the fly, as well as add new classes, members, etc. Then build onto it a way to create "code templates" so that you can autogenerate more than just method stubs, such as creating fields with accessors. Eventually you have a Class Editor with a plug-in system for code generation templates. If i then add a way to annotate the design, probably by extending the XML doc format already in there, i have a nice system for keeping design and implementation together.
So far, my add-in can mimic the Object pane of the Object Browser. Tonight I hope to add the Member pane functionality, then comes Adding new classes/members.
GUI programming Nostalgia
Back in the day (read: 1991ish) I thought that programming GUIs must have at least as sophisticated an interface as the GUIs themselves. Shortly thereafter I got to spend a day with a color Next Slab. That short exposure to the Next Interface Builder seemed to prove that supposition.
Sure, it was only a couple of hours, and really, I wasn't in the drivers side of the demonstration, but watching a word processor being built in an hour or two with lots of visual drag and drop was amazing. And this wasn't just a Form editor, but you actually visually connected visual components to code components. Cool stuff. Unfortunately, NeXT didn't make it (ok, a good bit of NeXT tech is in Mac OS X). Nonetheless, an expectation was set.
So I tried my hand at MPW (Macintosh Programmers Workshop) under Mac OS 7.1. Ugh. Not what i'd call elegant. And visual? No chance. I tried ThinkC.. A little easier to understand, but still all sorts of calls to toolkits originally written in Pascal, and nothing visual about that development.
Then came the new kid in Mac Development, Metrowerks. In its Codewarrior development suite there was a tool called Powerplant. It helped build the interface and create code stubs to fill in. It was ok at the time, and in many ways, it's what's now standard in most form builders. But still, it seemed like a lot of typing to do something inherently visual
Still pining for NeXT, i became a BeBox developer as soon as it became available for sale. I was full of that kool-aid drinking enthusiasm that comes naturally to any Mac Geek. Everything about BeBox screamed "technically superior". The community was full of people whose hopes and dreams were used to being crushed: Mac Developers, NeXT developers, Amiga programmers. I even saw a guy with a Taligent Pink T-shirt at the developer conference.
The one thing that didn't enter my mind when thinking BeBox was "immature technology". And it was. Very cool, with much potential, but the OS was still being written as people started doing development of application. Lots of things changed rather radically under the hood. The edition of Codewarrior that came with it didn't even have Powerplant. There were stories about and Interface Builder coming, but when it came it wasn't revolutionary in any sense of the word.
Long story short: Development under BeOS was no more advanced than any other OS and I moved on from the platform after realizing that it wasn't going to make it, especially when Apple didn't buy BeOS and bought NeXT instead.
Last couple of years I've spent most of my time working on the server side, saying "Bah!" to non-browser GUIs.
Now I'm back in GUI land, this time with C# under Visual Studio .NET and, all things considered, it is probably one of the best out there. Even if it is only incrementally more advanced than other suites for GUI development I've worked with. It's fun (even lacking Emacs mode) and quite productive.
But I still think that the potential of Visual Programming has not even had it's surface scratched. Ho hum.
EnvDTE
I figured I might as well blog this, since otherwise, i'll just loose the information I've come across. Simply this exists to write down useful or cool things I come across coding.
The impetus right now is on documenting my forays into EnvDTE in Visual Studio .NET 2003. EnvDTE is the namespace containing all the IDE automation classes, which I must say so far proves to be the worst documented portion of anything .NET related. I mean the MSDN docs are not even in the same format as the rest of the MSDN docs, the only examples in those docs are sparse at best and in VB.. Did I mention, I'm tackling this from the C# end.
Well, I made some headway so far and it's amazingly powerful stuff in there, but it's certainly a form of black magic. Nobody said it'd be easy.