DataSQRL Compiles Data Pipelines

Pick an Example:

IMPORT datasqrl.tutorials.clickstream.Click;  -- Import data
/* Find next page visits within 10 minutes */
VisitAfter := SELECT b.url AS beforeURL, a.url AS afterURL,
    a.timestamp AS timestamp
    FROM Click b JOIN Click a ON b.userid=a.userid AND
        b.timestamp < a.timestamp AND
        b.timestamp >= a.timestamp - INTERVAL 10 MINUTE;
/* Recommend pages that are frequently co-visited */
Recommendation := SELECT beforeURL AS url, afterURL AS rec,
    count(1) AS frequency FROM VisitAfter
    GROUP BY url, rec ORDER BY url ASC, frequency DESC;

Step 1: Implement in SQL

Develop a recommendation engine for your customers based on visits to your website.

DataSQRL builds on the SQL you already know for a low learning curve.

type Query {
  Recommendation(url: String!): [Recommendation!]
}

type Recommendation {
  url: String!
  rec: String!
  frequency: Int!
}

Step 2: Define API

Specify the GraphQL schema for the API. That's how external or internal customers access the processed data.

Step 3: Compile to Pipeline

DataSQRL compiles SQL + GraphQL schema into an optimized data pipeline integrating Apache Flink, Kafka, Postgres, and API layer into a robust, scalable, and easy to maintain data product.

AWS

GCP

Azure

Docker

Kubernetes

Confluent Cloud

Step 4: Deploy Anywhere

DataSQRL builds optimized executables for each component that run efficiently on your preferred cloud, managed service, or self-hosted.

Use the services and technologies you already trust to run your data pipeline.

Try Out this Example!

Saves You Time

DataSQRL allows you to focus on your data processing by eliminating the data plumbing that strangles your data pipeline implementation with busywork: data mapping, schema management, data modeling, error handling, data serving, API generation, and so on.

Easy to Use

Implement your data processing with the SQL you already know. DataSQRL allows you to focus on the "what" and worry less about the "how". Import your functions when SQL is not enough - DataSQRL makes custom code integration easy.

Fast & Efficient

DataSQRL builds efficient data pipelines that optimize data processing, partitioning, index selection, view materialization, denormalization, and scalability. There actually is some neat technology behind this buzzword bingo.