你好,游客 登录
背景:
阅读新闻

开源PaaS平台Cloud Foundry在部署实战

[日期:2017-07-06] 来源:  作者: [字体: ]

  Cloud Foundry介绍

  Cloud Foundry是业界的开源的PaaS。

  本文重点讲述Cloud Foundry在阿里云上的部署方案,对于Cloud Foundry介绍的部分简单略过。

  Cloud Foundry解决了什么问题?

  关键词:PaaS、开源

  Cloud Foundry是一种平台即服务(PaaS),兼容多种基础设施云,提供多种开发框架和应用服务。目前开源支持部署Cloud Foundry的基础设施云包括AWS、Azure、OpenStack等等,CF在开发框架上支持Java、.NET、Ruby等等,有很强的灵活性。

  IaaS帮助开发者和客户解决了基础设施的问题,Cloud Foundry作为PaaS,在云上让开发者的视角更高,开发者只需要关注应用和数据。

  

图片描述

 

  Cloud Foundry组件(v2版本)

  Cloud Foundry是由相对独立的多个模块构成的分布式系统,每个模块单独存在和运行,各模块之间通过消息机制进行通信。

  

图片描述

 

  Cloud Foundry目前最新的架构为v3版本,各个组件的功能,官网有详细的介绍,这边不再赘述。

  参考:http://docs.cloudfoundry.org/concepts/architecture/

  阿里云上部署Cloud Foundry实战

  环境说明:

  (本文使用软件版本和环境的说明,涉及到的概念下文会详细解释)

  阿里云region:华东1

  ruby 版本:2.3.0

  bosh-init 版本:0.0.96

  Bosh CLI 版本:1.3262.4.0

  CF CLI 版本:6.21.0

  部署流程

  在阿里云上部署Cloud Foundry,部署流程分为三大部分,分别为:

  1. 开通阿里云环境资源:包括准备阿里云账号,开通相关云产品资源等等

  2. 部署Bosh:安装bosh-init,使用bosh-init部署Bosh

  3. 部署Cloud Foundry:使用Bosh部署Cloud Foundry

  本文后续内容会根据三步部署流程详细讲述。

  一、开通阿里云环境资源

  Cloud Foundry需要部署在VPC环境下,需要开通VPC资源,并创建虚拟交换机,规划好部署Cloud Foundry的内网网段。

  同时,也需要创建一对Access Key ID和Access Key Secret,也可以使用现有的Access Key ID和Access Key Secret对。

  创建专有网络VPC

  网段没有限制,笔者选择了10.0.0./8网段

  

图片描述

 

  创建虚拟交换机

  可用区和网段没有限制,方便起见,笔者选择了华东1可用区E,网段为10.0.0.0/25

  

图片描述

 

  创建Access Key ID和Access Key Secret

  

图片描述

 

  二、部署Bosh

  Bosh介绍

  Bosh是一个统一了平台即服务软件(如Cloud Foundry)的发布、部署和生命周期管理的自动化配置部署工具。简单的说,Bosh的作用之一就是部署Cloud Foundry,部署Cloud Foundry之前,首先要部署Bosh。

  在讲怎么在阿里云上部署Bosh之前,首先需要了解Bosh中的几个基本概念。

  Bosh官网参考:https://bosh.io/docs

  CPI

  CPI全称Cloud Platform Interface,是Bosh对外开放的一组标准化接口,在IaaS上部署Bosh,需要实现这组接口,Bosh用CPI对IaaS的资源进行管理,包括创建虚拟机,释放虚拟机,等等……

  在阿里云上部署Bosh,需要用阿里云OpenAPI实现CPI。

  CPI参考:https://bosh.io/docs/cpi-api-v1.html

  CPI API v1完整接口列表:

  image

  bosh-init

  bosh-init是Bosh官网提供的一个开源工具,它的作用就是用来部署Bosh。

  bosh-init参考:https://bosh.io/docs/using-bosh-init.html

  stemcell

  stemcell是虚拟机镜像,预装了部署过程中需要的组件(其中最重要的组件是Bosh Agent),官方的描述是:A stemcell is a versioned Operating System image wrapped with IaaS specific packaging. Bosh用CPI创建的虚拟机,用stemcell镜像启动。

  stemcell参考:https://bosh.io/docs/stemcell.html

  release

  release是Bosh中一个安装部署包的概念,包含了所有安装分布式系统需要的源代码、配置文件、脚本文件等等,官网描述为:A release is a versioned collection of configuration properties, configuration templates, start up scripts, source code, binary artifacts, and anything else required to build and deploy software in a reproducible way.

  例如,安装Bosh,我需要一个Bosh-release;用Bosh安装Cloud Foundry,我需要一个cf-release。

  所有release都有版本迭代,都可以在Bosh官网找到。

  release参考:https://bosh.io/docs/release.html

  Deployment

  一个Deployment是一组VM的集合,由指定的stemcell镜像启动,用于部署对应release的系统。官方描述为:A deployment is a collection of VMs, built from a stemcell, that has been populated with specific releases and disks that keep persistent data. These resources are created based on a manifest file in the IaaS and managed by the BOSH Director, a centralized management server.

  在Bosh的概念里,一个Deployment对应一个release,Deployment的具体配置,写在Deployment manifest里,在部署过程中提供。

  Deployment参考:https://bosh.io/docs/deployment.html

  阿里云上部署Bosh

  1. 创建ECS

  通过阿里云控制台创建一个ECS(包年包月、按量均可),用于安装bosh-init。方便起见,下文以bosh-init指代这台ECS。

  推荐配置:

  规格:2核4G及以上

  镜像:Ubuntu 14.04 64位

  系统盘:40GB以上,高效云盘

  网络类型:VPC实例,选择在上一步创建好的VPC和虚拟交换机,公网IP可以选择不分配。

  

图片描述

 

  2. 给bosh-init配置公网IP

  因为bosh-init是VPC实例,需要给bosh-init绑定弹性公网IP,让bosh-init可以通过公网访问。

  按需购买弹性公网IP,并给bosh-init绑定弹性公网IP

  

图片描述

 

  3. 安装bosh-init

  参考文档:http://bosh.io/docs/install-bosh-init.html

  SSH登陆到bosh-init这台ECS

  下载bosh-init,下载地址见:http://bosh.io/docs/install-bosh-init.html

  执行权限

  chmod +x ~/Downloads/bosh-init-*

  移动到/usr/local/bin

  sudo mv ~/Downloads/bosh-init-* /usr/local/bin/bosh-init

  验证安装成功

  bosh-init -v

  安装对应环境,笔者使用的是Ubuntu的机器

  sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3

  确保已经安装Ruby 2+环境

  

image

 

  4. 下载bosh-release

  bosh-release是开源的,可以从Bosh官网下载:

  https://bosh.io/releases/github.com/cloudfoundry/bosh?all=1

  笔者使用的是255.3版本的bosh-release

  

图片描述

 

  下载cpi-release

  CPI近期会开源,请关注:

  https://github.com/alibaba/opstools

  配置manifest

  根据以下模板,新建一个manifest文件bosh.yml,在模板中填充阿里云资源相关的内容。

  name: bosh

  releases:

  - name: bosh

  url: file:///root/downloads/bosh-255.3.tgz

  - name: bosh-aliyun-cpi

  url: file:///root/downloads/bosh-aliyun-cpi.tgz

  resource_pools:

  - name: vms

  network: private

  cloud_properties:

  instance_type: ecs.n4.large # <— 实例规格

  networks:

  - name: private

  type: manual

  subnets:

  - range: 10.0.0.0/8

  gateway: 10.0.0.1

  cloud_properties: {

  SecurityGroupId: SECURITY_GROUP_ID, # <— 安全组ID

  VSwitchId: VSWITCH-ID # <— 虚拟交换机ID

  }

  - name: public

  type: vip

  jobs:

  - name: bosh

  instances: 1

  templates:

  - {name: nats, release: bosh}

  - {name: redis, release: bosh}

  - {name: postgres, release: bosh}

  - {name: blobstore, release: bosh}

  - {name: director, release: bosh}

  - {name: health_monitor, release: bosh}

  - {name: registry, release: bosh}

  - {name: aliyun_cpi, release: bosh-aliyun-cpi}

  resource_pool: vms

  networks:

  - name: private

  static_ips: [10.0.0.2] # <— ECS内网IP

  default: [dns, gateway]

  - name: public

  static_ips: [STATIC_IP] # <— 弹性公网IP

  properties:

  nats: &nats

  address: 127.0.0.1

  user: nats

  password: nats-password

  redis:

  listen_address: 127.0.0.1

  address: 127.0.0.1

  password: redis-password

  postgres: &db

  listen_address: 127.0.0.1

  host: 127.0.0.1

  user: postgres

  password: postgres-password

  database: bosh

  adapter: postgres

  registry: ®istry

  address: 127.0.0.1

  host: 127.0.0.1

  db: *db

  http: {user: admin, password: admin, port: 25777}

  username: admin

  password: admin

  port: 25777

  blobstore: &blobstore

  address: 127.0.0.1

  port: 25250

  provider: dav

  director: {user: director, password: director-password}

  agent: {user: agent, password: agent-password}

  director:

  address: 127.0.0.1

  name: my-bosh

  db: *db

  cpi_job: aliyun_cpi

  max_threads: 10

  user_management:

  provider: local

  local:

  users:

  - {name: admin, password: admin}

  - {name: hm, password: hm-password}

  hm:

  director_account: {user: hm, password: hm-password}

  resurrector_enabled: true

  aliyun: &aliyun

  access_key_id: ACCESS_KEY_ID # <--- 阿里云 Access Key ID

  access_key: ACCESS_KEY # <--- 阿里云 Access Key Secret

  default_key_name: bosh

  default_security_groups: [bosh]

  region_id: cn-hangzhou # <--- 阿里云 Region

  ntp: &ntp [0.pool.ntp.org, 1.pool.ntp.org]

  cloud_provider:

  template: {name: aliyun_cpi, release: bosh-aliyun-cpi}

  mbus: “https://mbus:[email protected]:6868” # <— ECS内网IP

  properties:

  aliyun: *aliyun

  agent:

  mbus: “nats://nats:[email protected]:4222” # <— ECS内网IP

  blobstore:

  provider: “dav”

  options:

  endpoint: “http://10.0.0.2:25250” # <— ECS内网IP

  user: “agent”

  password: “agent-password”

  blobstore: {provider: local, path: /var/vcap/micro_bosh/data/cache}

  ntp: *ntp

  nats: *nats

  registry: *registry

  blobstore: *blobstore

  部署Bosh

  执行部署命令:

  bosh-init deploy bosh.yml

  一个部署过程的示例:

  

图片描述

 

  8. 验证Bosh

  我们用Bosh CLI验证Bosh是否部署成功。

  Bosh CLI是Bosh官方提供的,用于和Bosh交互的命令行工具。在部署完成Bosh之后,用Bosh CLI和Bosh交互,执行相关命令,进行下一步Cloud Foundry的部署。

  安装Bosh CLI

  Bosh CLI可以安装在任意一台ECS上,或者本地主机上。

  安装Bosh CLI:https://bosh.io/docs/bosh-cli.html

  验证Bosh

  执行Bosh CLI命令:

  bosh target 10.0.0.2

  笔者的Bosh CLI安装在同一个安全组的ECS上,因此可以和部署Bosh的ECS进行内网通讯,直接通过内网IP,target到Bosh。如果需要通过公网通讯,需要给部署Bosh的ECS绑定弹性公网IP,或者使用NAT网关产品,保证网络能通。

  如图显示,成功连接到目标Bosh,验证Bosh成功。

  bosh releases、bosh stemcells这两条命令,使用Bosh部署Cloud Foundry的时候会用到,我们下节细讲。

  

图片描述

 

  三、部署Cloud Foundry

  基本概念

  首先我们也需要了解用Bosh部署Cloud Foundry过程中的几个基本概念。

  Bosh CLI

  上一节讲到,我们需要使用已经部署成功的Bosh来部署Cloud Foundry,通过Bosh CLI和Bosh进行交互,执行相关部署命令。因此,使用Bosh部署Cloud Foundry之前,首先需要了解Bosh CLI命令的使用。

  Bosh CLI几个基本命令:

  1.连接到指定Bosh

  bosh target [DIRECTOR_URL]

  2.列出当前release仓库中所有的release

  bosh releases

  3.上传release到Bosh的release仓库,只有仓库里的release可以用于部署

  bosh upload release [RELEASE_FILE]

  4.列出当前stemcell仓库里的所有stemcell

  bosh stemcells

  5.上传stemcell到Bosh的stemcell仓库,只有仓库里的stemcell可以用于部署

  bosh upload stemcell STEMCELL_PATH

  bosh upload stemcell STEMCELL_URL

  6.列出当前所有的Deployment

  bosh deployments

  7.列出当前Deployment的信息

  bosh deployment

  8.切换到指定manifest对应的Deployment

  bosh deployment [MANIFEST_PATH]

  9.执行部署当前Deployment

  bosh deploy

  Bosh CLI官网参考:http://bosh.io/docs/sysadmin-commands.html

  CPI

  在上一步部署Bosh中,已经部署好的Bosh中已经包含了CPI组件,因此在部署Cloud Foundry中不再需要CPI-release

  stemcell

  和上一步部署Bosh中类似。

  cf-release

  Cloud Foundry官网提供的压缩包,包含Cloud Foundry所有组件的源码。

  Deployment manifest

  Deployment的配置文件,主要描述了用哪个stemcell,用哪个cf-release,需要部署哪些CF组件,需要多少VM,VM规格信息,VM的IP信息,具体哪个VM部署哪个CF组件,等等……

  部署Cloud Foundry流程

  1. 下载cf-release

  cf-release是开源的,可以从Bosh官网下载:

  http://bosh.io/releases/github.com/cloudfoundry/cf-release?all=1

  笔者部署用的是215版本的cf-release

  

image

 

  登陆到安装有Bosh CLI的机器

  我们需要用Bosh CLI和Bosh进行交互,执行部署Cloud Foundry命令

  用Bosh CLI连接到部署好的Bosh

  bosh target 10.0.0.2

  上传cf-release

  上传下载完成的cf-release,执行以下Bosh CLI命令:

  bosh upload release

  

图片描述

 

  配置manifest

  根据以下模板,新建一个manifest文件cf.yml,在模板中填充阿里云资源相关的内容。

  name: ali-cf

  director_uuid: BOSH_DIRECTOR_UUID # <— Bosh Director UUID

  releases:

  - {name: cf, version: 215}

  networks:

  - name: private

  type: manual

  subnets:

  - range: 10.0.0.0/8

  gateway: 10.0.0.1

  #dns: [10.0.0.2]

  reserved: [“10.0.0.2”]

  static: [“10.0.0.3 - 10.0.0.100”]

  cloud_properties:

  SecurityGroupId: SECURITY_GROUP_ID # <— 安全组ID

  VSwitchId: VSWITCH_ID # <— 虚拟交换机ID

  resource_pools:

  - name: small_ecs

  network: private

  cloud_properties:

  instance_type: ecs.n1.tiny # <— 实例规格

  compilation:

  workers: 1

  network: private

  reuse_compilation_vms: true

  cloud_properties:

  instance_type: ecs.n1.medium # <— 实例规格

  update:

  canaries: 1

  max_in_flight: 1

  serial: false

  canary_watch_time: 30000-600000

  update_watch_time: 5000-600000

  jobs:

  - name: nats

  instances: 1

  resource_pool: small_ecs

  templates:

  - {name: nats, release: cf}

  networks:

  - name: private

  static_ips: [10.0.0.20] # <— ECS内网IP

  name: nfs

  instances: 1

  persistent_disk: 50

  resource_pool: small_ecs

  templates:

  {name: debian_nfs_server, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.21] # <— ECS内网IP

  name: postgres

  instances: 1

  persistent_disk: 50

  resource_pool: small_ecs

  templates:

  {name: postgres, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.22] # <— ECS内网IP

  update:

  serial: true

  name: cloud_controller

  instances: 2

  resource_pool: small_ecs

  templates:

  {name: cloud_controller_ng, release: cf}

  {name: cloud_controller_worker, release: cf}

  {name: cloud_controller_clock, release: cf}

  {name: nfs_mounter, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.23, 10.0.0.33] # <— ECS内网IP

  propertis:

  nfs_server:

  address: 10.0.0.21

  allow_from_entries: [10.0.0.0/24]

  name: hm9000

  instances: 1

  resource_pool: small_ecs

  templates:

  - {name: consul_agent, release: cf}

  {name: hm9000, release: cf}

  - {name: metron_agent, release: cf}

  {name: route_registrar, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.24] # <— ECS内网IP

  name: doppler_z1

  instances: 1

  resource_pool: small_ecs

  templates:

  {name: doppler, release: cf}

  networks:

  name: private

  properties:

  doppler: {zone: z1}

  doppler_endpoint:

  shared_secret: PASSWORD

  name: loggregator_trafficcontroller_z1

  instances: 1

  resource_pool: small_ecs

  templates:

  {name: loggregator_trafficcontroller, release: cf}

  {name: metron_agent, release: cf}

  {name: route_registrar, release: cf}

  networks:

  name: private

  properties:

  traffic_controller: {zone: z1}

  route_registrar:

  routes:

  name: doppler

  registration_interval: 20s

  port: 8081

  uris:

  “doppler.REPLACE_WITH_SYSTEM_DOMAIN”

  name: loggregator

  registration_interval: 20s

  port: 8080

  uris:

  ”loggregator.REPLACE_WITH_SYSTEM_DOMAIN”

  name: uaa

  instances: 1

  resource_pool: small_ecs

  templates:

  {name: uaa, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.25] # <— ECS内网IP

  properties:

  login:

  catalina_opts: -Xmx768m -XX:MaxPermSize=256m

  uaa:

  admin:

  client_secret: PASSWORD

  batch:

  password: PASSWORD

  username: batch_user

  cc:

  client_secret: PASSWORD

  scim:

  userids_enabled: false

  users:

  [email protected]|Cps123456|scim.write,scim.read,openid,cloud_controller.admin

  uaadb:

  address: 10.0.0.22 # <— postgres组件内网IP

  databases:

  {name: uaadb, tag: uaa}

  db_scheme: postgresql

  port: 5524

  roles:

  {name: uaaadmin, password: uaa-password, tag: admin}

  name: router

  instances: 1

  resource_pool: small_ecs

  templates:

  {name: gorouter, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.27] # <— ECS内网IP

  properties:

  dropsonde: {enabled: true}

  name: dea_ng

  instances: 1

  resource_pool: small_ecs

  templates:

  {name: dea_next, release: cf}

  networks:

  name: private

  static_ips: [10.0.0.26] # <— ECS内网IP

  properties:

  networks: {apps: private}

  app_domains: [DOMAIN] # <— domain

  domain: DOMAIN # <— domain

  system_domain: DOMAIN # <— domain

  system_domain_organization: default_organization

  cc:

  allow_app_ssh_access: false

  bulk_api_password: PASSWORD

  db_encryption_key: PASSWORD

  default_running_security_groups: [public_networks, dns]

  default_staging_security_groups: [public_networks, dns]

  install_buildpacks:

  - {name: java_buildpack, package: buildpack_java}

  - {name: ruby_buildpack, package: buildpack_ruby}

  - {name: nodejs_buildpack, package: buildpack_nodejs}

  - {name: go_buildpack, package: buildpack_go}

  - {name: python_buildpack, package: buildpack_python}

  - {name: php_buildpack, package: buildpack_php}

  - {name: staticfile_buildpack, package: buildpack_staticfile}

  - {name: binary_buildpack, package: buildpack_binary}

  internal_api_password: PASSWORD

  quota_definitions:

  default:

  memory_limit: 102400

  non_basic_services_allowed: true

  total_routes: 1000

  total_services: -1

  security_group_definitions:

  - name: private

  rules: []

  srv_api_uri: http://api.DOMAIN # <— domain

  staging_upload_password: PASSWORD

  staging_upload_user: staging_upload_user

  ccdb:

  address: 10.0.0.22 # <— postgres组件内网IP

  databases:

  - {name: ccdb, tag: cc}

  db_scheme: postgres

  port: 5524

  roles:

  - {name: ccadmin, password: cc-password, tag: admin}

  databases:

  databases:

  - {name: ccdb, tag: cc, citext: true}

  - {name: uaadb, tag: uaa, citext: true}

  port: 5524

  roles:

  - {name: ccadmin, password: cc-password, tag: admin}

  - {name: uaaadmin, password: uaa-password, tag: admin}

  dea_next:

  advertise_interval_in_seconds: 5

  heartbeat_interval_in_seconds: 10

  etcd:

  machines: [10.0.0.24] # <— hm9000组件内网IP

  hm9000:

  url: http://hm9000.DOMAIN # <— domain

  nats:

  machines: [10.0.0.20] # <— nats组件内网IP

  password: nats-password

  port: 4222

  user: nats

  nfs_server:

  no_root_squash: true

  address: 10.0.0.21 # <— nfs组件内网IP

  allow_from_entries: [10.0.0.0/24]

  uaa:

  no_ssl: true

  clients:

  gorouter:

  authorities: clients.read,clients.write,clients.admin,route.admin,route.advertise

  authorized-grant-types: client_credentials,refresh_token

  scope: openid,cloud_controller_service_permissions.read

  secret: PASSWORD

  cloud_controller_username_lookup:

  authorities: scim.userids

  authorized-grant-types: client_credentials

  secret: PASSWORD

  login:

  authorities: oauth.login,scim.write,clients.read,notifications.write,critical_notifications.write,emails.write,scim.useridsassword.write

  authorized-grant-types: authorization_code,client_credentials,refresh_token

  override: true

  redirect-uri: http://login.DOMAIN:8080/auth/cloudfoundry/callback # <— domain

  scope: openid,oauth.approvals,cloud_controller.read,cloud_controller.write

  secret: PASSWORD

  url: http://uaa.DOMAIN # <--- domain

  新建cf.yml对应的Deployment

  bosh deployment cf.yml

  执行部署命令

  bosh deploy

  验证Cloud Foundry

  部署完成之后,执行以下命令,查看Cloud Foundry部署详情:

  bosh vms

  

图片描述

  至此,Cloud Foundry在阿里云上部署成功。

推荐 打印 | 录入: | 阅读:
相关新闻      
本文评论   
评论声明
  • 尊重网上道德,遵守中华人民共和国的各项有关法律法规
  • 承担一切因您的行为而直接或间接导致的民事或刑事法律责任
  • 本站管理人员有权保留或删除其管辖留言中的任意内容
  • 本站有权在网站内转载或引用您的评论
  • 参与本评论即表明您已经阅读并接受上述条款