容器部署

容器部署 #

极限网关支持容器方式部署,可以运行在 K8s 集群环境。

安装演示 #

下载镜像 #

极限网关的镜像发布在 Docker 的官方仓库,地址如下:

https://hub.docker.com/r/medcl/infini-gateway

使用下面的命令即可获取最新的容器镜像:

docker pull medcl/infini-gateway:latest

验证镜像 #

将镜像下载到本地之后,可以看到极限网关的容器镜像非常小,只有不到 25MB,所以下载的速度应该是非常快的。

✗ docker images              
REPOSITORY                                      TAG       IMAGE ID       CREATED          SIZE
medcl/infini-gateway                            latest    fdae74b64e1a   47 minutes ago   23.5MB

创建配置 #

现在需要创建一个配置文件 gateway.yml,来进行基本的配置,如下:

entry:
  - name: es_gateway #your gateway endpoint
    enabled: true
    router: default
    network:
      binding: 0.0.0.0:8000
      reuse_port: true #you can start multi gateway instance, they share same port, to full utilize system's resources
    tls:
      enabled: false #if your es is using https, the gateway entrypoint should enable https too

flow:
  - name: cache_first
    filter: #comment out any filter sections, like you don't need cache or rate-limiter
      - name: get_cache
      - name: elasticsearch
        parameters:
          elasticsearch: prod  #elasticsearch configure reference name
          max_connection: 1000 #max tcp connection to upstream, default for all nodes
          max_response_size: -1 #default for all nodes
          balancer: weight
          refresh: # refresh upstream nodes list, need to enable this feature to use elasticsearch nodes auto discovery
            enabled: true
            interval: 30s
      - name: set_cache
  - name: request_logging # this flow is used for request logging, refer to `router`'s `tracing_flow`
    filter:
      - name: request_path_filter
        parameters:
          must_not: # any match will be filtered
            prefix:
              - /favicon.ico
      - name: request_logging
        parameters:
          queue_name: request_logging


router:
  - name: default
    tracing_flow: request_logging #a flow will execute after request finish
    default_flow: cache_first
    rules: #rules can't be conflicted with each other, will be improved in the future
      - method:
          - "*"
        pattern:
          - /
        flow:
          - cache_first # after match, which processing flow will go through

elasticsearch:
- name: prod
  enabled: true
  endpoint: http://192.168.3.201:9200 # if your elasticsearch is using https, your gateway should be listen on as https as well
  discovery: # auto discovery elasticsearch cluster nodes
    enabled: true
    refresh:
      enabled: true
  basic_auth: #used to discovery full cluster nodes, or check elasticsearch's health and versions
    username: elastic
    password: pass


modules:
- name: elastic
  enabled: true
  elasticsearch: prod
  init_template: true
- name: pipeline
  enabled: true
  runners:
    - name: primary
      enabled: true
      max_go_routine: 1
      threshold_in_ms: 0
      timeout_in_ms: 5000
      pipeline_id: request_logging_index


pipelines:
- name: request_logging_index
  start:
    joint: json_indexing
    enabled: true
    parameters:
      index_name: "gateway_requests"
      elasticsearch: "prod"
      input_queue: "request_logging"
      timeout: "60s"
      worker_size: 1
      bulk_size_in_mb: 1 #in MB
  process: []

queue:
  min_msg_size: 1
  max_msg_size: 104857600
  max_bytes_per_file: 53687091200
  sync_every_records: 100000 # sync by records count
  sync_timeout_in_ms: 10000 # sync by time in million seconds
  read_chan_buffer: 0

Note: 修改上面配置里面的 Elasticsearch 的相关配置,改成实际的连接地址和认证信息:

elasticsearch:
- name: prod
  enabled: true
  endpoint: http://192.168.3.201:9200 # if your elasticsearch is using https, your gateway should be listen on as https as well
  discovery: # auto discovery elasticsearch cluster nodes
    enabled: true
    refresh:
      enabled: true
  basic_auth: #used to discovery full cluster nodes, or check elasticsearch's health and versions
    username: elastic
    password: pass

启动网关 #

使用如下命令启动极限网关容器:

docker run -p 2900:2900 -p 8000:8000  -v=`pwd`/gateway.yml:/gateway.yml  medcl/infini-gateway:latest

验证网关 #

如果都运行正常的话,应该可以看到如下的信息:

➜  /tmp docker run -p 2900:2900 -p 8000:8000  -v=`pwd`/gateway.yml:/gateway.yml  medcl/infini-gateway:latest
   ___   _   _____  __  __    __  _       
  / _ \ /_\ /__   \/__\/ / /\ \ \/_\ /\_/\
 / /_\///_\\  / /\/_\  \ \/  \/ //_\\\_ _/
/ /_\\/  _  \/ / //__   \  /\  /  _  \/ \ 
\____/\_/ \_/\/  \__/    \/  \/\_/ \_/\_/ 

[GATEWAY] A light-weight, powerful and high-performance elasticsearch gateway.
[GATEWAY] 1.0.0_SNAPSHOT, b61758c, Mon Dec 28 14:32:02 2020 +0800, medcl, no panic by default
[12-30 05:26:41] [INF] [instance.go:24] workspace: data/gateway/nodes/0
[12-30 05:26:41] [INF] [runner.go:59] pipeline: primary started with 1 instances
[12-30 05:26:41] [INF] [entry.go:257] entry [es_gateway] listen at: http://0.0.0.0:8000
[12-30 05:26:41] [INF] [app.go:247] gateway now started.
[12-30 05:26:45] [INF] [reverseproxy.go:196] elasticsearch [prod] endpoints: [] => [192.168.3.201:9200]

如果希望容器运行在后台,加上 -d 参数,如下:

docker run -d -p 2900:2900 -p 8000:8000  -v=`pwd`/gateway.yml:/gateway.yml  medcl/infini-gateway:latest

使用命令行或者浏览器访问地址: http://localhost:8000/ 应该就能正常访问 Elasticsearch 了,如下:

➜  /tmp curl -v http://localhost:8000/
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
> GET / HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: INFINI
< Date: Wed, 30 Dec 2020 05:12:39 GMT
< Content-Type: application/json; charset=UTF-8
< Content-Length: 480
< UPSTREAM: 192.168.3.201:9200
< 
{
  "name" : "node1",
  "cluster_name" : "pi",
  "cluster_uuid" : "Z_HcN_6ESKWicV-eLsyU4g",
  "version" : {
    "number" : "6.4.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "04711c2",
    "build_date" : "2018-09-26T13:34:09.098244Z",
    "build_snapshot" : false,
    "lucene_version" : "7.4.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host localhost left intact
* Closing connection 0

Docker Compose #

还可以使用 docker compose 来管理容器实例,新建一个 docker-compose.yml 文件如下:

version: "3.5"

services:
  infini-gateway:
    image: medcl/infini-gateway:latest
    ports:
      - 2900:2900
      - 8000:8000
    container_name: "infini-gateway"
    volumes:
      - ../gateway.yml:/gateway.yml

volumes:
  dist:

在配置文件所在目录,执行如下命令即可启动,如下:

➜  docker-compose up
Starting infini-gateway ... done
Attaching to infini-gateway
infini-gateway    |    ___   _   _____  __  __    __  _       
infini-gateway    |   / _ \ /_\ /__   \/__\/ / /\ \ \/_\ /\_/\
infini-gateway    |  / /_\///_\\  / /\/_\  \ \/  \/ //_\\\_ _/
infini-gateway    | / /_\\/  _  \/ / //__   \  /\  /  _  \/ \ 
infini-gateway    | \____/\_/ \_/\/  \__/    \/  \/\_/ \_/\_/ 
infini-gateway    | 
infini-gateway    | [GATEWAY] A light-weight, powerful and high-performance elasticsearch gateway.
infini-gateway    | [GATEWAY] 1.0.0_SNAPSHOT, b61758c, Mon Dec 28 14:32:02 2020 +0800, medcl, no panic by default
infini-gateway    | [12-30 13:24:16] [INF] [instance.go:24] workspace: data/gateway/nodes/0
infini-gateway    | [12-30 13:24:16] [INF] [api.go:244] api server listen at: http://0.0.0.0:2900
infini-gateway    | [12-30 13:24:16] [INF] [runner.go:59] pipeline: primary started with 1 instances
infini-gateway    | [12-30 13:24:16] [INF] [entry.go:257] entry [es_gateway] listen at: http://0.0.0.0:8000
infini-gateway    | [12-30 13:24:16] [INF] [app.go:247] gateway now started.

离线部署 #