This guide describes how to use the kfctl command line interface (CLI) todeploy Kubeflow on GCP. The command line deployment gives you more control overthe deployment process and configuration than you get if you use the deploymentUI. If you’re looking for a simpler deployment procedure, see how to deployKubeflow using the deployment UI.

Before you start

Before installing Kubeflow on the command line:

Follow these steps to download the kfctl binary for the Kubeflow CLI and set some handy environment variables: Download the kfctl v1.0.2 release from the kfctl releases page. Unpack the tar ball: tar -xvf kfctlv1.0.2.tar.gz Log in. You only need to run this command once: gcloud auth login Create user credentials.

1. Ensure you have installed the following tools:

   • kubectl.
   • gcloud. If you already have gcloudinstalled, run gcloud components update toget the latest version of all your installed Cloud SDK components.

2. If you’re usingCloud Shell, enableboost mode.

3. Make sure that your GCP project meets the minimum requirementsdescribed in the project setup guide.

4. Follow the guidesetting up OAuth credentials.to create OAuth credentials for Cloud Identity-Aware Proxy (CloudIAP).

Prepare your environment

Follow these steps to download the kfctl binary for the Kubeflow CLI and setsome handy environment variables:

1. Download the kfctl v1.0.2 release from thekfctl releases page.

2. Unpack the tar ball:

3. Log in. You only need to run this command once:

4. Create user credentials. You only need to run this command once:

5. Configure gcloud default values for zone and project

   • kfctl by default uses the gcloud defaults for zone and project
   • You can override this by explicitly setting zone and project in your KFDeffile

6. Select the KFDef spec to use as the basis for your deployment

7. Create environment variables containing the OAuth client ID and secret that you created earlier

   • The CLIENT_ID and CLIENT_SECRET can be obtained from the Cloud Console by selectingAPIs & Services -> Credentials

8. Pick a name KF_NAME for your Kubeflow deployment and directory foryour configuration.

   • For example, your kubeflow deployment name might be ‘my-kubeflow’ or ‘kf-test’.
   • Set base directory where you want to store one or more Kubeflow deployments.For example, ${HOME}/kf_deployments.

9. (Optional) Add the kfctl binary to your path. If you don’t add kfctl to your path, you must use the full patheach time you run kfctl.

Notes:

• ${PROJECT} - The project ID of the GCP project where you want Kubeflowdeployed.

• ${ZONE} - The GCP zone where you want to create the Kubeflow deployment.You can see a list of zones in theCompute Engine documentation.If you plan to use accelerators, you must choose a zone that supports thetype you want. See the guide tocustomizing your Kubeflow deployment.

• ${CONFIG_URI} - The GitHub address of the configuration YAML file thatyou want to use to deploy Kubeflow. For GCP deployments, the recommendedconfiguration is:

  When you run kfctl apply or kfctl build (see the next step), kfctl createsa local version of the configuration YAML file which you can furthercustomize if necessary.

• ${KF_NAME} - The name of your Kubeflow deployment.If you want a custom deployment name, specify that name here.For example, my-kubeflow or kf-test.The value of KF_NAME must consist of lower case alphanumeric characters or‘-', and must start and end with an alphanumeric character.The value of this variable cannot be greater than 25 characters. It mustcontain just a name, not a directory path.You also use this value as directory name when creating the directory whereyour Kubeflow configurations are stored, that is, the Kubeflow applicationdirectory.

• ${KF_DIR} - The full path to your Kubeflow application directory.

Deploying Kubeflow

To deploy Kubeflow using the default settings,run the kfctl apply command:

kfctl will try to populate the KFDef spec with various defaults automatically

• project and zone will be set based on your gcloud config defaults
• the name for the deployment will be inferred from the directory ${KF_DIR}
• You can override these values by modifying your KFDef spec before running the build and applycommands

You can follow the instructions in the next section to override these defaults.

Customizing your Kubeflow deployment

The process outlined in the previous step configures Kubeflow with various defaults.You can follow the instructions below to have greater control.

1. Download the KFDef file to your local directory to allow modifications

   • CONFIG_FILE should be the name you would like to use for your local config file; e.g. “kfdef.yaml”

2. Edit the KFDef spec in the yaml file. The following snippet shows you how to set values in the configuration fileusing yq:

   • PROJECT: The GCP project to deploy in
   • ZONE: The zone to deploy in
   • KF_NAME: The name used for your deployment.

3. Run the kfctl build command to generate kustomize and GCP Deployment manager configuration files for your deployment:

4. To customize your GKE cluster modify the deployment manager configuration filesin the directory ${KF_DIR}/gcp_config.

   • For more information refer to:
     • deployment manager docs.

5. To customize individual Kubeflow applications modify the Kustomize manifests in the directory${KF_DIR}/kustomize

   • For more information please refer to the kustomize docs.

6. Run the kfctl apply command to deploy Kubeflow:

Check your deployment

Follow these steps to verify the deployment:

1. The deployment process creates a separate deployment for your data storage.After running kfctl apply you should notice two newdeployments:

   • {KF_NAME}-storage: This deployment has persistent volumes for yourpipelines.
   • {KF_NAME}: This deployment has all the components of Kubeflow, includinga GKE clusternamed ${KF_NAME} with Kubeflow installed.

2. When the deployment finishes, check the resources installed in the namespacekubeflow in your new cluster. To do this from the command line, first setyour kubectl credentials to point to the new cluster:

   Then see what’s installed in the kubeflow namespace of your GKE cluster:

Access the Kubeflow user interface (UI)

Follow these steps to access the Kubeflow central dashboard:

1. Enter the following URI into your browser address bar. It can take 20minutes for the URI to become available:

   You can run the following command to get the URI for your deployment:

   The following command sets an environment variable named HOST to the URI:

2. Follow the instructions on the UI to create a namespace. See the guide tocreation of profiles.

Notes:

• It can take 20 minutes for the URI to become available.Kubeflow needs to provision a signed SSL certificate and register a DNSname.
• If you own or manage the domain or a subdomain withCloud DNSthen you can configure this process to be much faster.See kubeflow/kubeflow#731.

Understanding the deployment process

This section gives you more details about the kfctl configuration anddeployment process, so that you can customize your Kubeflow deployment ifnecessary.

kfctl process and configuration

The kfctl deployment process includes the following commands:

• kfctl build - (Optional) Creates configuration files defining the variousresources in your deployment. You only need to run kfctl build if you wantto edit the resources before running kfctl apply. See the guide tocustomizing your Kubeflow deployment.
• kfctl apply - Creates or updates the resources.
• kfctl delete - Deletes the resources.

The kfctl deployment process applies default values to certain propertiesas follows:

• Email address: kfctl attempts to fetch your email address from yourCloud SDK configuration. You can run gcloud config list to see the defaultemail address, which the command output lists as the account.If kfctl can’t find a valid email address, you must use theflag --email <your email address> to pass a valid email address. This emailaddress becomes an administrator in the configuration of your Kubeflowdeployment.

• GCP project ID: kfctl attempts to fetch your project ID from yourCloud SDK configuration. You can run gcloud config list to see youractive project ID.

• GCP zone: kfctl attempts to fetch the zone from your Cloud SDKconfiguration. You can run gcloud config list to see your active zone.

• Kubeflow deployment name: kfctl defaults to the name of the directorywhere you run the kfctl build or kfctl apply command.

You can also explicitly set the following values in your ${CONFIG_FILE}configuration file:

• Kubeflow deployment name
• GCP project
• GCP zone
• Email address

The following snippet shows you how to set values in the configuration fileusing yq:

Application layout

Your Kubeflow application directory ${KF_DIR} contains the following files anddirectories:

• ${CONFIG_FILE} is a YAML file that defines configurations related to yourKubeflow deployment.

  • This file is a copy of the GitHub-based configuration YAML file thatyou used when deploying Kubeflow:kfctl_gcp_iap.v1.0.0.yaml.
  • When you run kfctl apply or kfctl build, kfctl createsa local version of the configuration file, ${CONFIG_FILE},which you can further customize if necessary.

• gcp_config is a directory that containsDeployment Manager configuration filesdefining your GCP infrastructure.

  • The directory is created when you run kfctl build or kfctl apply.
  • You can modify these configurations to customize your GCP infrastructure.After modifying a configuration, run kfctl apply again.

• kustomize is a directory that contains the kustomize packages for Kubeflowapplications. Seehow Kubeflow uses kustomize.

  • The directory is created when you run kfctl build or kfctl apply.
  • You can customize the Kubernetes resources by modifying the manifests andrunning kfctl apply again.

We recommend that you check in the contents of your ${KF_DIR} directoryinto source control.

GCP service accounts

The kfctl deployment process creates three service accounts in yourGCP project. These service accounts follow the principle of leastprivilege.The service accounts are:

• ${KF_NAME}-admin is used for some admin tasks like configuring the loadbalancers. The principle is that this account is needed to deploy Kubeflow butnot needed to actually run jobs.
• ${KF_NAME}-user is intended to be used by training jobs and models to accessGCP resources (Cloud Storage, BigQuery, etc.). This account has a much smallerset of privileges compared to admin.
• ${KF_NAME}-vm is used only for the virtual machine (VM) service account. Thisaccount has the minimal permissions needed to send metrics and logs toStackdriver.

Download Kubectl For Macos Download

Basic authentication (deprecated)

Next steps

Download Kubectl For Mac

• Run a full ML workflow on Kubeflow, using theend-to-end MNIST tutorial or theGitHub issue summarization Pipelinesexample.
• See how to delete your Kubeflow deploymentusing the CLI.
• See how to customize your Kubeflowdeployment.
• See how to upgrade Kubeflow and how toupgrade or reinstall a Kubeflow Pipelinesdeployment.
• Troubleshoot any issues you mayfind.

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Download Kubectl For Mac Os 10

Sorry to hear that. Please tell us how we can improve.

Download Kubectl For Macos Windows 7

Download Kubectl For Mac Os 1