One Half Done

After a week of coding, i finally have a working MTOM unwrapper. Getting this was relatively easy, and involved parsing the message for the boundary, seperating the message parts, recombining them and encoding the binary octets as base64. See the code below if you are interested. Included is also a simple wrapper for the SOAP4r Libary so I can use it through rails.
http://ross-assoc.net/ruby4mtom.rb

Next up is figuring out a way to hook the MTOM handler into the generation of the SOAP message. The main thing to know is which parts are defined/restricted to base64 or hex, otherwise it is very hard to tell what needs to be seperated out into parts and un-encoded.

Any thoughts or bugs are much appreciated.

More Thoughts

Spend some of today doing more research on the feasibility of implementing MTOM in ruby using the rails framework. As expected, there really isn't any action happening with MTOM outside of the Java and .NET worlds. But I did find a lot of interesting analysis of the MTOM spec that should be a big help. First off, this article was quite helpful:

http://www.devx.com/xml/Article/34797

I was originally pretty worried that MTOM includes some weird formatting of the HTTP header, which I wouldn't have access to using the rails framework, but except for the content-type (which one can get access to through the @headers instance variable in the ActionController class), the HTTP header is essentially the same. FYI - for setting a unique HTTP content type in rails, check out this article. That means that the transformation occurs on the HTTP body only.

Basically, there are four things to set/read in a SOAP message to indicate that MTOM has been used:

1. The application/xop+xml media is present
2. The media type of the HTTP message is multipart/related.
3. The media type of the root part of the MIME Multipart/Related package is application/xop+xml.
4. The start-info parameter indicates a content type of application/soap+xml.

The other really interesting thing I found out is that MTOM can be implemented on top of SOAP 1.1. These guys at IBM came up with a white paper on how to indicate an MTOM'd message using SOAP 1.1. I haven't had a chance to read it in depth, but it looks promising (and gives me an excuse to drop the SOAP1.2 part of this project :).

One more thing I'm noticing is that there aren't any MTOM test tools or validators out there. That is going to make testing any implementation I do in ruby pretty difficult (unless I want to set up a Java endpoint myself!) If anyone out there is interested in coding a basic Java/.NET MTOM endpoint, shoot me an email: mjreich at gmail dot com.

Direction

Life got crazy (moved to a new house), and I haven't been able to spend as much time on this as I would like. So, here's a dump of where I'm thinking for this project.

As I get into this more and more, it seems like the SOAP1.2 part of this project is pretty ambitious. Given the NaHi (the main developer of SOAP4R) hasn't touched SOAP 1.2 yet, I'm skeptical that I'll be able to integrate anything nicely into his library that will handle the new spec. That said, for my project, I've been able to generate stub code from the SOAP4R library, which means that we aren't doing anything that is technically new in SOAP1.2. Additionally, I'm really only interested in the MTOM attachment mechanism (now that DIME is officially DOA), not the down and dirty details of SOAP. This all leads me to think that the value added here would be to create a wrapper for the SOAP4R library that runs a transform on the incoming and outgoing messages and:

1. Changes the namespaces to indicate SOAP1.2 messages and trigger the correct parser on the receiving end.
2. Makes any other changes to a SOAP 1.1 message to make it semantically acceptable to a SOAP 1.2 parser.
3. Extracts the Base64Encoded message parts, optimizes them and creates the correct envelope structure (the actual implementation of the MTOM specification).

Again, from the specification, I'm not sure what the functional relationship between MTOM and SOAP1.2 is, other than they were both released at the same time. Given that MTOM and XOP operate outside of the SOAP message spec, and don't alter the non Base64Encoded parts in any appreciable way, making a message completely compliant with the SOAP1.2 spec may not be necessary at all (which would reduce the amount of work on my end by an order of magnitude.)

Stay tuned: next up is creating the wrapper class and trying to clean this up so that it might be useful to someone on another project.

HTTP header content type

One thing I ran into problems with when testing was that Mongrel was returning a HTTP content type of application/xml rather than text/xml. After a little searching online, I found an easy (and pretty darn useful) fix. In the 'environment.rb' file in the config folder is a section for defining MIME types that returned through the rails 'respond_to' method. I simply added this line of code and got the result I was looking for:

Mime::Type.register "text/xml", :xml

Hope this helps!

One more thing: boot.rb

After doing some testing and getting this up on my hosted server, I found one problem that others have run into. When using SOAP4r as a gem, you may get an error along the lines of 'uninitialized constant SOAP::Mapping::EncodedRegistry(nameError)'. For some reason, the gem load order is wonky, so there is an easy fix involving boot.rb. Check out the link for more info:

http://dev.ctor.org/soap4r/wiki/RailsAndSoap4R

Ruby on Rails + SOAP4R server

And I thought it would be hard! After scouring the web and finding nothing on how to integrate the SOAP4r stub code onto rails (to create a SOAP server on top of the rails framework), I finally figured out a way to do it. The trick was in the server/servant files that are generated by wsdl2ruby; by diving down into the class definitions I was able to pull out the atomic components of the servant and stick them in a rails project. Here's the breakdown:

First, I created an empty rails project using the scaffold generator. I set it up with a base rails class that will act as a router to grab the post data and pass it through to the SOAP router and parser. What we are interested in is the 'create' method in the applicationController that rails generates (which using the REST architecture is where the HTTP POST command is captured.) I'll walk through the raw code first, but keep in mind that I plan to wrap all of this into a helper class for cleanliness.

def create
  @router = ::SOAP::RPC::Router.new('rubynode')
  servant = NetworkNodePortType2.new
  res = ''
  req = request

@router is the SOAP4R router that takes an input post dump and parses it using the type definitions created by WSDL2RUBY. The 'servant' is an instance of the basic class generated by the WSDL2RUBY script. The servant class contains the application logic for generating the return data from the parameters in the request message (this class is dummy generated by the WSDL2RUBY script, and must be filled in; check out the doc/literal example in the soap4r install folder for the specific example.) The 'res' variable will be used to hold the SOAP response message, but for now is nilled out, and the 'req' variable is a copy of the rails generated 'request' data structure that contains the raw POST dump and other HTTP header info needed later on.

  NetworkNodePortType2::Methods.each do definitions
    opt = definitions.last
    if opt[:request_style] == :document
      @router.add_document_operation(servant, *definitions)
    else
      @router.add_rpc_operation(servant, *definitions)
    end
  end

The above section of code is the intialization stub taken from the generated server app class. This simply initializes the operation definitions from the WSDL and binds them to the servant methods through the router. That is, the 'definitions' include the input and output message types and bindings for each operation defined in the WSDL. This code sets up the path to the application logic (in the servant class) when a defined message is recieved.

  conn_data = ::SOAP::StreamHandler::ConnectionData.new
  setup_req(conn_data, req)
  @router.external_ces = nil
  conn_data = @router.route(conn_data)
  input = setup_res(conn_data, req, res)

  respond_to do |format|
    format.xml { render :xml => input }
  end
end

def setup_req(conn_data, req)
  conn_data.receive_string = req.raw_post
  conn_data.receive_contenttype = req.content_type
  conn_data.soapaction = nil
end

def setup_res(conn_data, req, res)
  res = conn_data.send_string
end

Here's the meat of the code. First, the 'conn_data' variable contains the streamhandler connection data type, which is used to house the raw POST dump and the HTTP header info. Next, we map the 'request' information into the 'conn_data' structure and then pass it into the router. We then grab the response from the conn_data structure and pass it to rails XML handler to be returned.

And there you go: a decent first shot at integrating the SOAP4R library/generated stub code into a working Ruby on Rails web service implementation. Incidentally, a wrapper class using these basic tools would be a perfect place to put the MTOM hooks: all that would need to be done is grab the raw SOAP response message and run a transform on the 'base64encoded' parts.

Rails + SOAP4R != Fun

The next step of this project (now that I have figured out how to bind to the WSDL) is to put up a test service implementation of the WSDL and then validate it using either a SOAP 1.2 message validator, or barring that, a test client that a colleague is developing in .NET.

Consuming web services through rails is easy.

I'm learning that serving anything through rails is hard.

I may be able to use ActionWebService (I've heard from some that this uses soap4r behind the scenes), but my guess is not. So, next up is to figure out how to bind the soap4r servant stub to Rails - possibly through the native rest interface. SOAP operates over HTTP using the POST method, so if I can find a way to trigger the SOAP4R servant stub and pass it the input, and trap/send the output, this may work. Possibly through routes.rb.