Explainable. Repeatable. Scaleable.

Blog

Featured Article

Say Hello To Pachyderm: Hub

Today we’re excited to announce the public beta of our fully-managed and hosted offering of Pachyderm: Hub. When my co-founder Joey Zwicker and I started Pachyderm nearly 6 years ago, we set out to make data science better. Pachyderm’s current feature set provides collaboration on large scale data workloads, but it’s confined to companies with the resources to manage a Kubernetes cluster. We’ve felt for a long time, that the last part has been holding us back.

Read full article
Previous Posts

Say Hello To Pachyderm: Hub

Hosted and managed Pachyderm for those who want data lineage, without the hassle of managing infrastructure

Read Full Article

(Kubernetes as a Service) as a Service

What it's like to build kubernetes-as-a-service, as a service.

Read Full Article

New Case Study: Digital Reasoning

See how Digital Reasoning is preparing for the future with Pachyderm.

Read Full Article

Using Pachyderm to generate new Game of Thrones scripts

New Pachyderm Example that uses a RNN to genereate new Game Of Thrones scripts.

Read Full Article

4 Reasons to Get Excited About Pachyderm 2019

4 Reasons to Get Excited About Pachyderm 2019

Read Full Article
View all

Examples

For those who like to learn while doing.

OpenCV (aka "Hello World")

This tutorial walks you through the deployment of a Pachyderm pipeline to do simple edge detection on a few images.

View on GitHub

ML Pipeline for Tweet Generation (gpt-2)

In this example we'll create a machine learning pipeline that generates tweets using OpenAI's gpt-2 text generation model.

View on GitHub

Distributed hyperparameter tuning

This example demonstrates how you can evaluate a model or function in a distributed manner on multiple sets of parameters.

View on GitHub

Spouts

This example connects to an IMAP mail account, collects all the incoming mail and analyzes it for positive or negative sentiment, sorting the emails into directories in its output repo with scoring information added to the email header "X-Sentiment-Rating"

View on GitHub

Mnist with TFJob and Pachyderm

This example uses the canonical mnist dataset, Kubeflow, TFJobs, and Pachyderm to demonstrate an end-to-end machine learning workflow with data provenance.

View on GitHub

Create a Join Pipeline

In this example, we will create a join pipeline. A join pipeline executes your code on files that match a specific naming pattern.

View on GitHub
View all