One Half Done

After a week of coding, i finally have a working MTOM unwrapper. Getting this was relatively easy, and involved parsing the message for the boundary, seperating the message parts, recombining them and encoding the binary octets as base64. See the code below if you are interested. Included is also a simple wrapper for the SOAP4r Libary so I can use it through rails.
http://ross-assoc.net/ruby4mtom.rb

Next up is figuring out a way to hook the MTOM handler into the generation of the SOAP message. The main thing to know is which parts are defined/restricted to base64 or hex, otherwise it is very hard to tell what needs to be seperated out into parts and un-encoded.

Any thoughts or bugs are much appreciated.

One more thing: boot.rb

After doing some testing and getting this up on my hosted server, I found one problem that others have run into. When using SOAP4r as a gem, you may get an error along the lines of 'uninitialized constant SOAP::Mapping::EncodedRegistry(nameError)'. For some reason, the gem load order is wonky, so there is an easy fix involving boot.rb. Check out the link for more info:

http://dev.ctor.org/soap4r/wiki/RailsAndSoap4R

Ruby on Rails + SOAP4R server

And I thought it would be hard! After scouring the web and finding nothing on how to integrate the SOAP4r stub code onto rails (to create a SOAP server on top of the rails framework), I finally figured out a way to do it. The trick was in the server/servant files that are generated by wsdl2ruby; by diving down into the class definitions I was able to pull out the atomic components of the servant and stick them in a rails project. Here's the breakdown:

First, I created an empty rails project using the scaffold generator. I set it up with a base rails class that will act as a router to grab the post data and pass it through to the SOAP router and parser. What we are interested in is the 'create' method in the applicationController that rails generates (which using the REST architecture is where the HTTP POST command is captured.) I'll walk through the raw code first, but keep in mind that I plan to wrap all of this into a helper class for cleanliness.

def create
  @router = ::SOAP::RPC::Router.new('rubynode')
  servant = NetworkNodePortType2.new
  res = ''
  req = request

@router is the SOAP4R router that takes an input post dump and parses it using the type definitions created by WSDL2RUBY. The 'servant' is an instance of the basic class generated by the WSDL2RUBY script. The servant class contains the application logic for generating the return data from the parameters in the request message (this class is dummy generated by the WSDL2RUBY script, and must be filled in; check out the doc/literal example in the soap4r install folder for the specific example.) The 'res' variable will be used to hold the SOAP response message, but for now is nilled out, and the 'req' variable is a copy of the rails generated 'request' data structure that contains the raw POST dump and other HTTP header info needed later on.

  NetworkNodePortType2::Methods.each do definitions
    opt = definitions.last
    if opt[:request_style] == :document
      @router.add_document_operation(servant, *definitions)
    else
      @router.add_rpc_operation(servant, *definitions)
    end
  end

The above section of code is the intialization stub taken from the generated server app class. This simply initializes the operation definitions from the WSDL and binds them to the servant methods through the router. That is, the 'definitions' include the input and output message types and bindings for each operation defined in the WSDL. This code sets up the path to the application logic (in the servant class) when a defined message is recieved.

  conn_data = ::SOAP::StreamHandler::ConnectionData.new
  setup_req(conn_data, req)
  @router.external_ces = nil
  conn_data = @router.route(conn_data)
  input = setup_res(conn_data, req, res)

  respond_to do |format|
    format.xml { render :xml => input }
  end
end

def setup_req(conn_data, req)
  conn_data.receive_string = req.raw_post
  conn_data.receive_contenttype = req.content_type
  conn_data.soapaction = nil
end

def setup_res(conn_data, req, res)
  res = conn_data.send_string
end

Here's the meat of the code. First, the 'conn_data' variable contains the streamhandler connection data type, which is used to house the raw POST dump and the HTTP header info. Next, we map the 'request' information into the 'conn_data' structure and then pass it into the router. We then grab the response from the conn_data structure and pass it to rails XML handler to be returned.

And there you go: a decent first shot at integrating the SOAP4R library/generated stub code into a working Ruby on Rails web service implementation. Incidentally, a wrapper class using these basic tools would be a perfect place to put the MTOM hooks: all that would need to be done is grab the raw SOAP response message and run a transform on the 'base64encoded' parts.

Rails + SOAP4R != Fun

The next step of this project (now that I have figured out how to bind to the WSDL) is to put up a test service implementation of the WSDL and then validate it using either a SOAP 1.2 message validator, or barring that, a test client that a colleague is developing in .NET.

Consuming web services through rails is easy.

I'm learning that serving anything through rails is hard.

I may be able to use ActionWebService (I've heard from some that this uses soap4r behind the scenes), but my guess is not. So, next up is to figure out how to bind the soap4r servant stub to Rails - possibly through the native rest interface. SOAP operates over HTTP using the POST method, so if I can find a way to trigger the SOAP4R servant stub and pass it the input, and trap/send the output, this may work. Possibly through routes.rb.

Diving Deeper

I've been diving into the SOAP4R code a little to get a lay of the land and figure out where any MTOM hooks would be possible. BTW, if you haven't checked out the ruby-debug gem, I highly recommend it. Here's a short screencast on basic useage (if you are familiar with gdb, then this is cake).

After running through the code a few times with the debugger, I have a basic sense as to what the wsdl2ruby.rb file is doing (and where it's generating errors). I've been using this document/literal WSDL which includes the SOAP12 schema (just to try out). Given that SOAP4R doesn't have support for SOAP1.2, I didn't really expect this to work - and it doesn't. Keeps bombing out when it is building the definitions:

ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}binding
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}operation
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}body
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}fault
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}address

Also, wsdl2ruby doesn't seem to like it when the service and definition elements are named the same: it is using the names of the elements to name the files, and runs into a collision when it creates the 'service' library because the definitions library (of the same name) has already been created. I'd be curious if there is anything in the WS-I profile (or WSDL spec for that matter) that prohibits naming the defs and service the same.

The other interesting tidbit is that if I replace the SOAP12 namespace to SOAP and rename the definitions element so that it does not conflict, wsdl2ruby binds fine and generates all the correct code without errors! So, I'm at least working with a WSDL that is bindable, aside from some minor details. However, I would like to keep the SOAP12 refs in there, as MTOM is technically only supported by SOAP12 (although I don't know of any technical issue in the MTOM/XOP spec that would preclude using it with SOAP11).

So, here's my overview of the WSDL2ruby.rb file (the version in lib\ruby\1.8\wsdl\soap, rather than the wrapper version included in the soap4r distribution. I'm running soap4r 1.5.6.

This file does two main things: 1) parses the WSDL using the XSD::XMLparser engine and, 2) dumps the class definitions, type mappings, and client/server stub code (including the driver and servant) into files. The WSDL parsing is the interesting part, and where it keeps barfing. Specifically, when trying to import the 'binding' element, it compares the imported element name with the the DefinitionsName variable and throws an unknown element error (line 115 of lib/ruby/1.8/wsdl/parser.rb). The value of DefinitionsName is 'definitions' which is obviously not 'binding'. Next up is figuring out where DefinitionsName comes from, or is defined, and what it means.

Kickoff

Welcome all! This blog will document my efforts to create/extend MTOM (Message Transmission and Optimization Mechanism) functionality in Ruby (mainly through the SOAP4R library). MTOM was added as a W3C recommendation with SOAP 1.2 back almost a year ago and was created to allow for inline attachment of payloads in a SOAP message, without having to hack the stack with DIME or SwA. MTOM (and the optimization component XOP) are pretty cool, as the user at either end sees only a unified infoset, with binary represented by the xsd:base64encoded type. There are no hooks, one simply has stick the binary data in a base64encoded element and MTOM/XOP takes over the rest. Currently, MTOM is implemented in .NET 2 and Xfire, thought I haven't explored any other toolkits, so it may be elsewhere too.

This blog will document my progress, and hopefully provide some needed assistance to people who are stuck down the black hole called ruby SOAP support.