Parsing massive XML documents using XMLReader

By
rp
– June 23, 2010Posted in: Web Development

I recently had the need to parse a massive (2gb) xml file and read it back to the database, the standard approach of SimpleXML and Dom was not going to work. I wanted to use XMLReader for this purpose. So here are the quick notes for helping anyone getting upto speed of what the quick start code looks like.

Quoting from its website:

The XMLReader extension is an XML Pull parser. The reader acts as a cursor going forward on the document stream and stopping at each node on the way.

// create the reader object
$reader = new XMLReader();

// reader the XML file.
$reader->open($abms_file);

// start reading the XML File.
while($reader->read()) {
    // take action based on the kind of node returned
   switch($reader->nodeType) {
       // read more http://uk.php.net/manual/en/class.xmlreader.php#xmlreader.constants.element
       case (XMLREADER::ELEMENT):
              // get the name of the node.
              $node_name  = $reader->name;
              // move the pointer to read the next item
              $reader->read();
              // action based on the $node_name
           break;
       case (XMLREADER::END_ELEMENT):
            // do something based on when the element closes.
            break;
   }
}

Pretty handy is this parsing method since it scales well for huge documents as well as for tiny documents and the memory footprint remains consistently the same.

Tags: , , , ,

About rp

Architect for large, highly scalable LAMP applications and Technical Manager with special focus on metrics based continuous improvement of teams and products. Rajat has close to a decade of experience of a very wide range of skills related to infrastructure, middleware, app servers all the way to front-end technologies and software development methodologies including agile, iterative waterfall, waterfall as well as ah-hoc startup using the right approach in the right context to reduce time to market.