Leveraging Datalab's Marker and Modal for Efficient Document Intelligence Deployment

Collaborative efforts between Modal and Datalab bring a notable advancement in document intelligence deployment, providing a streamlined method for developers and teams to use cutting-edge models. Datalab is known for its efficient document parsing solutions like Marker and Surya, both of which are designed to enhance document processing capabilities.

Marker is a specialized model with under one billion parameters, engineered specifically for understanding document layouts. Its design focuses on delivering deterministic, high-fidelity parsing, which notably reduces hallucinations seen in larger language models. This efficiency translates into significantly lower operational costs while ensuring accuracy, an aspect crucial for commercial applications. Marker and other tools from Datalab have achieved over 48,000 stars on GitHub, marking them as trusted resources among researchers, startups, and established enterprises.

Quick Deployment with Modal

With the integration of Modal, any developer can easily deploy Datalab’s Marker pipeline and the Surya OCR toolkit in under five minutes. For those looking to dive right in, cloning the Marker repository allows quick local setups, but deploying on Modal maximizes scalability. By following a straightforward process, users can provision GPU-powered containers to run Marker efficiently.

To get started, execute the following commands to install Modal and run the deployment example:

pip install modal
modal setup

git clone [email protected]:datalab-to/marker.git
cd marker/examples/
modal deploy marker_modal_deployment.py

Once deployed, can leverage Modal’s capabilities to build a document OCR web application, as outlined in this Modal example.

Modal offers an introductory credit of $30 per month for compute resources, providing ample resources to kick off OCR tasks without immediate costs.

Operational Mechanics

Behind the scenes, Modal enables swift deployment of the Marker model on powerful GPUs, allowing for high throughput by automatically scaling GPU resources for intensive batch processing. This infrastructure supports a seamless experience, as it eliminates lengthy cold starts through caching model weights in a Modal Volume. Through this mechanism, frequent model redownloads are avoided, ensuring rapid inference regardless of container location.

On invocation of the inference function, Modal instantly prepares a container that meets the specified environmental and hardware needs outlined in the function's decorator. This design philosophy streamlines configurations by embedding necessary setups directly within the application code, reducing reliance on external config files.

inference_image = modal.Image.debian_slim(python_version="3.12").uv_pip_install(
    "marker-pdf[full]==1.9.3", "torch==2.8.0"
)

@app.function(gpu="l40s", volumes=marker_cache, image=inference_image)
def parse_document(document: bytes, ...) -> str | dict:
    # Load Marker model from Volume and run
    ...

This approach extends its benefits for users needing to process bulk documents. Modal can autonomously scale up to thousands of GPUs based on incoming request volumes, ensuring rapid processing times without manual adjustments. Global capacity pools are designed to eliminate bottlenecks associated with quota limits.

Advantages of Using Marker

Marker shines in its support for over 90 languages and its adeptness at handling intricate tables and mathematical content extraction from PDFs. Users can apply Marker to diverse tasks, including:

• Building indexed PDF knowledge bases for retrieval-augmented generation (RAG)
• Parsing multilingual PDF contents for training purposes
• Extracting critical data from unstructured documents

For detailed benchmarks that demonstrate its merit in both accuracy and throughput metrics compared to competitors like Llamaparse and Mathpix, please refer to the Marker documentation.

Performance Efficiency

In performance-critical scenarios, Marker was intentionally designed to address real-world operational needs. It maintains a fine balance between high accuracy and substantial throughput. For example, typical processing rates on standard hardware yield around 0.22 pages per second, drastically improving to approximately 2.2 pages per second when run on Modal's infrastructure, leveraging potent hardware solutions such as the H100 GPU and employing optimizations like Flash Attention.

For scenarios where rapid processing of multiple PDFs is required, Modal's ability to autoscale is instrumental, permitting operations across hundreds of GPUs to achieve superior throughput. Commercial applications can further benefit from Datalab’s API enhancements, producing throughput rates between three to four pages per second.

Conclusion: Elevating Document Intelligence

Modal's collaboration with Datalab marks a significant step forward for users seeking to deploy efficient, state-of-the-art document intelligence solutions. With intuitive access to powerful tools like Marker and Surya, teams can now easily implement best practices in document processing. Begin your journey with this example, and explore the sophisticated capabilities of these tools today.