Please enable JavaScript.
Coggle requires JavaScript to display documents.
Kubeflow Pipelines - Coggle Diagram
Kubeflow Pipelines
-
SDK
-
dsl
domain-specific lang
contains useful decorators for
P, components, etc
components
handles indivual pipe units - components.
Contains things like func_to_container_op,
load_container_from_file (container from config file)
load_container_from_uri (prebuilt, from some repo)
extensions
Custom cloud stuff like TPU / GPU, K8S secrets, etc
diagnose_me
kfp.cli.diagnose_me.dev_env
reports on diagnostic metadata
from your development environment,
such as your python library version.
kfp.cli.diagnose_me.kubernetes_cluster
reports on diagnostic data from your Kubernetes cluster,
such as Kubernetes secrets.
-
compiler
creates .yaml config file from the
pipeline. It is
converted by KF service into
K8S suitable resource spec
-
-
SDK deep dive
-
Component I/O
Parameters
By value
int, double, float, or str
By file
complex entitites, datasets
-
-
Artifacts
heavy-weight datasets,
models, etc
-
Output
If your component returns multiple outputs by value, annotate your function with the typing.NamedTuple type hint and use the collections.namedtuple function to return your function’s outputs as a new subclass of tuple
-
-
Notebooks
1 Nb server = 1 namespace =
= 1 project group, team, etc
Main Components
MPI operator
idea
provide framework-agnostic distributed training platform
works with Horovod, PyTorch, TF, Caffee, MXNet
-
-
-