分类目录归档:php

Jupyter Notebook

对于编程初学者,如果有一个开箱即用的环境,比如web页面,就可以进行编程交互,那是极友好。有时候我们想在远程服务器上执行一些脚本,输出一些结果,比如科学计算;有时候又想在服务器上执行一些命令但又不能直接登录服务器,如果能够在web界面上操作或作为跳板机,那也是极友好的。Jupyter Notebook是基于IPython的一个基于web交互执行的在线环境,支持Python,也支持其他编程语言,比如Julia和R。所创建Notebook文档可以自动保存执行过的代码、结果,方便进行回放。
Jupyter Notebok的安装很方便,可以使用Anaconda来安装,或者手动安装。Python3下手动安装,

pip3 install jupyter
export PATH=$PATH:/usr/local/python3/bin

查看一下

[root@localhost local]# pip3 show jupyter
Name: jupyter
Version: 1.0.0
Summary: Jupyter metapackage. Install all the Jupyter components in one go.
Home-page: http://jupyter.org
Author: Jupyter Development Team
Author-email: jupyter@googlegroups.org
License: BSD
Location: /usr/local/python3/lib/python3.7/site-packages
Requires: jupyter-console, notebook, ipywidgets, nbconvert, qtconsole, ipykernel
Required-by: 

如果直接运行jupyter notebook,那么会生成一个本地可以访问的带token的url,每次都不一样,不是很方便。设置密码,以便登录

[root@localhost opt]# jupyter notebook password
Enter password: 
Verify password: 
[NotebookPasswordApp] Wrote hashed password to /root/.jupyter/jupyter_notebook_config.json
[root@localhost bin]# cat /root/.jupyter/jupyter_notebook_config.json 
{
  "NotebookApp": {
    "password": "sha1:e04153005102:961b12eef91987a06b497f915fc3f18c62d8f714"
  }

由于是在虚拟机里面,我们并不需要Jupyter自动打开浏览器,但需要监听来自任意IP的请求,指定端口9030。这里使用root用户运行Jupyter,默认是不允许的:

[root@localhost opt]# jupyter notebook --no-browser --allow-root --ip 0.0.0.0 --port 9030
[I 02:13:44.320 NotebookApp] Serving notebooks from local directory: /opt
[I 02:13:44.320 NotebookApp] The Jupyter Notebook is running at:
[I 02:13:44.320 NotebookApp] http://(localhost.localdomain or 127.0.0.1):9030/
[I 02:13:44.320 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 02:13:59.664 NotebookApp] 302 GET / (192.168.33.1) 1.22ms
[I 02:14:23.597 NotebookApp] Kernel started: 7ad63717-7a65-4dec-9d5a-9af654c28f75
[I 02:14:25.204 NotebookApp] Adapting to protocol v5.1 for kernel 7ad63717-7a65-4dec-9d5a-9af654c28f75
[I 02:14:37.350 NotebookApp] Starting buffering for 7ad63717-7a65-4dec-9d5a-9af654c28f75:ea68853b742c40f8bcf8745529ea95de
[I 02:14:43.735 NotebookApp] Kernel started: 5b569c8d-6936-4bd2-9674-0317c46948f6
[I 02:14:44.124 NotebookApp] Adapting to protocol v5.0 for kernel 5b569c8d-6936-4bd2-9674-0317c46948f6
[2019-06-03 02:14:43] kernel.DEBUG: Connection settings {"processId":6751,"connSettings":{"shell_port":39990,"iopub_port":48184,"stdin_port":40113,"control_port":43426,"hb_port":49075,"ip":"127.0.0.1","key":"d5f89bba-890ecf15e6b20718411170ad","transport":"tcp","signature_scheme":"hmac-sha256","kernel_name":"jupyter-php"},"connUris":{"stdin":"tcp://127.0.0.1:40113","control":"tcp://127.0.0.1:43426","hb":"tcp://127.0.0.1:49075","shell":"tcp://127.0.0.1:39990","iopub":"tcp://127.0.0.1:48184"}} []
[2019-06-03 02:14:44] KernelCore.DEBUG: Initialized sockets {"processId":6751} []

然后打开浏览器,访问http://192.168.33.70:9030,输入账号密码,就可以在web里面运行Python了

Jupyter默认带了SQL扩展,使用ipython-sql来执行,只需要安装对应的驱动,这里使用PyMySQL

python3 -m pip install PyMySQL

然后在Web里面执行就可以了

Jupyter还有其他扩展,参考这里
除了可以执行Python和SQL,Jupyter Notebook也可以支持其他语言,在这里列出了。通常执行方式是通过Bash执行,或者通过ZeroMQ来通信,参考这里实现。一个Jupyter的kernal将需要监听以下几个socket:

  • Shell:执行命令
  • IOPub:推送执行结果
  • Stdin:接收输入
  • Control:接收控制命令,比如关闭、终端
  • Heartbeat:心跳检测
  • 这个思路也可以用来做IOT设备的远程监控,交互执行。
    这里安装一下PHP 7这个kernel,作者甚至还提供了installer,但是首先要安装ZeroMQ以便与Jupyter服务通信

    yum install php-pecl-zmq
    wget https://litipk.github.io/Jupyter-PHP-Installer/dist/jupyter-php-installer.phar
    ./jupyter-php-installer.phar install
    

    查看安装文件

    [root@localhost opt]# ls -la /usr/local/share/jupyter/kernels/
    total 0
    drwxr-xr-x. 4 root root 34 May 10 06:10 .
    drwxr-xr-x. 3 root root 20 May  9 07:30 ..
    drwxr-xr-x. 2 root root 24 May  9 07:30 jupyter-php
    drwxr-xr-x. 2 root root 40 May 10 06:10 lgo
    
    [root@localhost opt]# cat /usr/local/share/jupyter/kernels/jupyter-php/kernel.json 
    {"argv":["php","\/opt\/jupyter-php\/pkgs\/vendor\/litipk\/jupyter-php\/src\/kernel.php","{connection_file}"],"display_name":"PHP","language":"php","env":{}}
    

    这个扩展使用了react/zmq来监听Jupyter请求,使用psysh来交互执行PHP代码。

    如果想要更改Jupyter的web模板,可以在以下目录找到

    [root@localhost vagrant]# ls -la /usr/local/python3/lib/python3.7/site-packages/notebook/templates
    total 92
    drwxr-xr-x.  2 root root  4096 May  9 06:33 .
    drwxr-xr-x. 19 root root  4096 May  9 06:33 ..
    -rw-r--r--.  1 root root   147 May  9 06:33 404.html
    -rw-r--r--.  1 root root   499 May  9 06:33 browser-open.html
    -rw-r--r--.  1 root root  4258 May  9 06:33 edit.html
    -rw-r--r--.  1 root root   856 May  9 06:33 error.html
    -rw-r--r--.  1 root root  4256 May  9 06:33 login.html
    -rw-r--r--.  1 root root  1179 May  9 06:33 logout.html
    -rw-r--r--.  1 root root 23162 May  9 06:33 notebook.html
    -rw-r--r--.  1 root root  6559 May  9 06:33 page.html
    -rw-r--r--.  1 root root  1089 May  9 06:33 terminal.html
    -rw-r--r--.  1 root root 12130 May  9 06:33 tree.html
    -rw-r--r--.  1 root root   544 May  9 06:33 view.html
    

    Jupyter Notebook Web前端采用WebSocket与服务器交互,服务器接收消息并转发给对应的kernel执行或控制,并将结果推送给前端。Jupyter Notebook也可以直接打开Terminal,在远程服务器上执行命令。注意这里的用户就是刚才运行jupyter的用户

    许多web terminal也都是采用WebSocket来做交互,比如xterm.jswebtty
    Juypter Notebook适合单用户(单机)使用,如果提供多用户使用(比如教学),可以使用Jupyter Hub,可以使用docker快捷部署。

    参考链接:
    Jupyter Notebook Extensions
    Jupyter – How do I decide which packages I need?
    PsySH——PHP交互式控制台
    Jupyter项目
    WebSocket 教程

    Let’s encrypt

    最近域名主机双双到期了,原来的服务商建议域名迁出,于是转移到了Godaddy,过程很顺利。主机迁到了linode,一个是因为它便宜,另一个是因为想给自己的网站加个SSL证书。
    首先是服务器环境Apache, PHP, MaraiDB(MySQL)的配置。 linode 创建主机很简单,点点就好了,然后可以去启动机器,设置SSH访问。

    yum update
    yum install httpd php php-cli php-mbstring php-pdo php-mysql php-gd php-tidy
    

    启动Apache

    systemctl start httpd.service
    systemctl enable httpd.service
    

    然后直接访问你的服务器ip,可以看到默认的欢迎界面

    ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'
    curl 1.139.x.x
    

    CentOS 7目前默认不提供Mysql Server,而是Mariadb。MySQL的命令行仍然可以使用并兼容, PHP仍然可以使用PDO及MySQL扩展访问它。

    yum install mariadb-server mariadb
    systemctl start mariadb
    

    然后设置管理员密码及安全设置

    mysql_secure_installation
    

    开机启动Mariadb服务

    systemctl enable mariadb.service
    

    可以在/var/www/html目录下创建一个测试脚本来验证安装情况

    <?php
    phpinfo();
    

    WordPress则是从原本的数据库全部导出,文件全部打包回来。
    在Linode上创建对应的数据库,用户及导入脚本

    MariaDB [(none)]> CREATE database courages_wordpress;
    
    MariaDB [(none)]> CREATE USER courages_wp IDENTIFIED BY '*************';
    
    MariaDB [(none)]> grant all privileges on courages_wordpress.* to courages_wp@localhost identified by '*************';
    

    导入数据库

    mysql -uroot -p courages_wordpress < /tmp/wordpress.sql
    

    将文件解压并复制到/var/www/html目录

    tar -xzvf backup.tar.gz
    cp -R backup/public_html/* /var/www/html/*
    chown -R apache:apache /var/www/html
    

    更改Apache设置AllowOverride 为all,以便支持WordPress的链接重定向。

    vim /etc/httpd/conf/httpd.conf
    
    <Directory "/var/www/html">
        #
        # Possible values for the Options directive are "None", "All",
        # or any combination of:
        #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
        #
        # Note that "MultiViews" must be named *explicitly* --- "Options All"
        # doesn't give it to you.
        #
        # The Options directive is both complicated and important.  Please see
        # http://httpd.apache.org/docs/2.4/mod/core.html#options
        # for more information.
        #
        Options Indexes FollowSymLinks
    
        #
        # AllowOverride controls what directives may be placed in .htaccess files.
        # It can be "All", "None", or any combination of the keywords:
        #   Options FileInfo AuthConfig Limit
        #
        AllowOverride All
    
        #
        # Controls who can get stuff from this server.
        #
        Require all granted
    </Directory>
    

    重启Apache

    systemctl restart httpd.service
    

    在linode的DNS manager那里新增一个新的domain,在服务器列表里面选中对应的服务器就可以了,然后就可以看到对应的域名解析信息。
    域名转移会要求一个key,从原注册商那里解锁并获得,在Godday输入Key后,它会发邮件与你确认,然后将DNS域名服务器改为linode的域名服务器就好了。

    Let’s Encrypt提供免费90天的SSL证书,如果证书到期了就需要再次更新下。如果你有shell权限,它推荐使用Cerbot来安装和更新证书。CentOS 7 + Apache的安装非常简单。首先安装EPEL源,要不然找不到对应的安装包

    yum install epel-release
    yum install certbot-apache
    certbot --authenticator webroot --installer apache
    

    设置一下域名,网站目录及域名重定向

    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Plugins selected: Authenticator webroot, Installer apache
    Enter email address (used for urgent renewal and security notices) (Enter 'c' to
    cancel): <email@example.com>
    Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
    
    -------------------------------------------------------------------------------
    Please read the Terms of Service at
    https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
    agree in order to register with the ACME server at
    https://acme-v01.api.letsencrypt.org/directory
    -------------------------------------------------------------------------------
    (A)gree/(C)ancel: A
    
    -------------------------------------------------------------------------------
    Would you be willing to share your email address with the Electronic Frontier
    Foundation, a founding partner of the Let's Encrypt project and the non-profit
    organization that develops Certbot? We'd like to send you email about EFF and
    our work to encrypt the web, protect its users and defend digital rights.
    -------------------------------------------------------------------------------
    (Y)es/(N)o: Y
    Starting new HTTPS connection (1): supporters.eff.org
    No names were found in your configuration files. Please enter in your domain
    name(s) (comma and/or space separated)  (Enter 'c' to cancel): courages.us
    Obtaining a new certificate
    Performing the following challenges:
    http-01 challenge for courages.us
    Input the webroot for courages.us: (Enter 'c' to cancel): /var/www/html
    Waiting for verification...
    Cleaning up challenges
    
    We were unable to find a vhost with a ServerName or Address of courages.us.
    Which virtual host would you like to choose?
    (note: conf files with multiple vhosts are not yet supported)
    -------------------------------------------------------------------------------
    1: ssl.conf                       |                       | HTTPS | Enabled
    -------------------------------------------------------------------------------
    Press 1 [enter] to confirm the selection (press 'c' to cancel): 1
    Deploying Certificate for courages.us to VirtualHost /etc/httpd/conf.d/ssl.conf
    
    Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
    -------------------------------------------------------------------------------
    1: No redirect - Make no further changes to the webserver configuration.
    2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
    new sites, or if you're confident your site works on HTTPS. You can undo this
    change by editing your web server's configuration.
    -------------------------------------------------------------------------------
    Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
    Created redirect file: le-redirect-courages.us.conf
    Rollback checkpoint is empty (no changes made?)
    
    -------------------------------------------------------------------------------
    Congratulations! You have successfully enabled https://courages.us
    
    You should test your configuration at:
    https://www.ssllabs.com/ssltest/analyze.html?d=courages.us
    -------------------------------------------------------------------------------
    

    在浏览器访问一下域名即有小绿钥匙。可以在/etc/httpd/conf.d/ssl.conf查看相应的SSL证书配置

    <VirtualHost _default_:443>
    ServerName courages.us
    SSLCertificateFile /etc/letsencrypt/live/courages.us/cert.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/courages.us/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateChainFile /etc/letsencrypt/live/courages.us/chain.pem
    </VirtualHost>
    

    由于证书在90天后即将失效,可以加入crontab自动更新

    certbot renew --dry-run
    crontab -e
    
    0 0,12 * * * python -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew 
    

    最近登录后台发现WordPress提示升级到PHP 7.3,按照它的指示

    在CentOS 7上升级PHP5.4 到PHP 7.3很简单:
    首先安装Remi和EPEL仓库

    yum install wget
    wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    wget http://rpms.remirepo.net/enterprise/remi-release-7.rpm
    rpm -Uvh remi-release-7.rpm 
    rpm -Uvh epel-release-latest-7.noarch.rpm
    
    yum install yum-utils
    

    启用remi-php73的源,yum update升级会自动升级PHP及扩展

    [root@li846-239 ~]# yum-config-manager --enable remi-php73
    [root@li846-239 ~]# yum repolist
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirrors.linode.com
     * epel: kartolo.sby.datautama.net.id
     * extras: mirrors.linode.com
     * remi-php73: mirror.xeonbd.com
     * remi-safe: mirror.xeonbd.com
     * updates: mirrors.linode.com
    repo id                                                                                                   repo name                                                   status
    base/7/x86_64                                                                                             CentOS-7 - Base                                              10,019
    epel/x86_64                                                                                               Extra Packages for Enterprise Linux 7 - x86_64               13,051
    extras/7/x86_64                                                                                           CentOS-7 - Extras                                               385
    remi-php73                                                                                                Remi's PHP 7.3 RPM repository for Enterprise Linux 7 - x86_64   305
    remi-safe                                                                                                 Safe Remi's RPM repository for Enterprise Linux 7 - x86_64    3,188
    updates/7/x86_64                                                                                          CentOS-7 - Updates                                            1,511
    repolist: 28,825
    
    
    yum update -y
    

    检查PHP版本,重启Apache

    [root@li846-239 ~]# php -v
    PHP 7.3.4 (cli) (built: Apr  2 2019 13:48:50) ( NTS )
    Copyright (c) 1997-2018 The PHP Group
    Zend Engine v3.3.4, Copyright (c) 1998-2018 Zend Technologies
    [root@li846-239 ~]# systemctl restart httpd
    

    也可以禁用对应PHP版本的源,选择性升级PHP到对应版本

    yum-config-manager --disable remi-php72
    

    顺便升级下php-mcrypt和ZipArchive

    yum install php-mcrypt
    yum install php-pecl-zip
    

    参考链接:
    How To Install Linux, Apache, MySQL, PHP (LAMP) stack On CentOS 7
    How to enable EPEL repository?
    How to Secure Your Server
    Introduction to FirewallD on CentOS
    How to Upgrade PHP 5.6 to PHP 7.2 on CentOS VestaCP

    日志系统设计

    日志系统是项目开发/运维当中非常重要的一部分,提供产品使用情况,跟踪调试信息等。以前都是各个项目各自实现日志记录,比如PHP、JAVA各自实现一套,如果要跨项目/服务器进行查询/跟踪/统计,则比较麻烦,比如Web后台–>业务模块–>基础组件,客户端–>公共接口–>业务模块等等。
    目前的项目都是将日志写入本地,首先需要定义统一的规范:

    格式
    DateTime | ServerIP | ClientIP | PID | RequestID|  Type | Level | Message|  Code
    
    解释
    DateTime:记录日志的当前时间戳
    ServerIP:当前服务器IP
    ClientIP:客户端IP
    PID:进程ID
    RequestID:交易号,即请求的唯一标识,使用uniqid(+UserCode)或UUID或,一次请求会有多条日志,,用于关联本次请求内的相关日志
    Type:日志类型,比如统计,操作(审计),业务等
    Level:日志等级
    Message:日志内容,当为值为数组或者对象时转为JSON
    Type为RUNTIME时,表示运行日志,属性:自定义
    Type为HTTP时,表示来源请求,属性:Url,Method(Get/Post),Params,RemoteIP,UserAgent,ReferUrl[,Action,Method]
    Type为REST时,表示外部调用,属性:Type(Http/Https),Url,Port,RequestParams,Response,RunTime;
    Type为SQL时,表示SQL执行,属性:Sql,RunTime
    Code:标识码,记录错误码、响应码、统计码、版本、自定义信息,方便统计
    

    这里的RequestID由入口处自动产生,用于标识一次请求,关联所产生的所有日志,需要在各个项目之间传递。为了减少日志,Level通常为Info级别,避免产生过多日志。为了方便调试追踪,RequestID和Level也可以在其他参数中指明,比如HTTP头里面附加。
    然后是日志收集:客户端收走日志系统,发送给日志系统服务端。
    然后分析处理呈现:服务端将接收到的日志,发给处理其他组件分析处理,提供Web界面的查询系统。研发人员,可以错误信息,定位问题;获悉程序运行情况进行调优;大数据分析日志,得出产品使用情况;运维平台则可以进行业务报警。

    日志产生由个语言依照规范自行实现,收集、保持则由FlumeKafka实现。Flume是一个分布式的日志收集、合并、移动系统,能够监控文件变化,将变化部分传输出去。Kafka是一个分布式的发布/订阅的消息流平台,包括Broker,Consumer,Producer都支持分布式,依赖Zookeeper实现。

    在PHP上面的实现,一开始使用log4php,看起来很美好,但是性能很差,对业务影响较大。决定再次简化,砍掉不必要的东西(Socket,邮件等等),在C语言开发的PHP日志扩展SeasLog基础上在做开发,将日志文件保存在本地。为了减少日志所占用内存,每超过一定的大小的日志即进行保存,否则在最后进行保存,利用了Nginx的fastcgi_finish_request特性。生产上发现,每天产生的文件日志太大了,需要控制日志信息大小、等级,并及时清理。
    对于Web后台,还结合FirePHP,将日志直接输出到浏览器,方便边运行变调试。

    参考链接:
    最佳日志实践
    Optimal Logging
    Flume+Kafka收集Docker容器内分布式日志应用实践
    基于Flume的美团日志收集系统(一)架构和设计
    Twitter是如何构建高性能分布式日志的
    开源日志系统比较
    Kafka剖析(一):Kafka背景及架构介绍
    基于Flume的野狗实时日志系统的演进和优化
    EVODelavega/phpkafka
    有赞统一日志平台初探
    RabbitMQ和kafka从几个角度简单的对比
    Flume-ng的原理和使用
    利用flume+kafka+storm+mysql构建大数据实时系统

    使用Docker Compose管理Docker容器

    上一篇创建了一个PHP的运行环境,现在需要一个MySQL服务,直接运行:

    root@thinkpad:~# docker run --name rocket-mysql -v /home/rocketfish/mysql/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=root -d mysql:5.7
    

    就自动下载并得到一个mysql版本为5.7的服务镜像。
    这时候,要运行一个Web服务我们需要运行两次的docker run才可以,如果还有更多的Web容器或其他的服务容器呢?
    Docker官方提倡一个容器仅提供一个服务,多个服务/容器可以使用docker-compose来管理。
    docker-compose本身是一个Python写的工具,可以直接通过pip安装:

    root@thinkpad:~# sudo pip install --upgrade pip
    

    如果你本地并没有Python环境,也可以采用docker-compose的docker镜像来运行:

    root@thinkpad:/home/compose-web# curl -L https://github.com/docker/compose/releases/download/1.8.0/run.sh > /usr/local/bin/docker-compose
    root@thinkpad:/home/compose-web# docker-compose --version
    #开始下载镜像,建议用第一种
    

    查看帮助

    root@thinkpad:~# docker-compose -h
    Define and run multi-container applications with Docker.
    
    Usage:
      docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]
      docker-compose -h|--help
    
    Options:
      -f, --file FILE             Specify an alternate compose file (default: docker-compose.yml)
      -p, --project-name NAME     Specify an alternate project name (default: directory name)
      --verbose                   Show more output
      -v, --version               Print version and exit
      -H, --host HOST             Daemon socket to connect to
    
      --tls                       Use TLS; implied by --tlsverify
      --tlscacert CA_PATH         Trust certs signed only by this CA
      --tlscert CLIENT_CERT_PATH  Path to TLS certificate file
      --tlskey TLS_KEY_PATH       Path to TLS key file
      --tlsverify                 Use TLS and verify the remote
      --skip-hostname-check       Don't check the daemon's hostname against the name specified
                                  in the client certificate (for example if your docker host
                                  is an IP address)
    
    Commands:
      build              Build or rebuild services
      bundle             Generate a Docker bundle from the Compose file
      config             Validate and view the compose file
      create             Create services
      down               Stop and remove containers, networks, images, and volumes
      events             Receive real time events from containers
      exec               Execute a command in a running container
      help               Get help on a command
      kill               Kill containers
      logs               View output from containers
      pause              Pause services
      port               Print the public port for a port binding
      ps                 List containers
      pull               Pulls service images
      push               Push service images
      restart            Restart services
      rm                 Remove stopped containers
      run                Run a one-off command
      scale              Set number of containers for a service
      start              Start services
      stop               Stop services
      unpause            Unpause services
      up                 Create and start containers
      version            Show the Docker-Compose version information
    

    这些命令包括了创建/启动/停止/暂停/继续运行容器。
    首先要创建docker-compose.yml,这是一个YAML格式的文档

    mkdir docker
    cd docker
    mkdir web
    mkdir db
    vim docker-compose.yml
    

    内容如下:

    version: '2'
    services:
      db:
        image: mysql:5.7
        ports:
        - "3306:3306"
        restart: always
        environment:
          MYSQL_ROOT_PASSWORD: root
          MYSQL_DATABASE: web
          MYSQL_USER: root
          MYSQL_PASSWORD: rootweb
        volumes:
        - ./db/data:/var/lib/mysql
      web:
        depends_on:
          - db
        image: nginx-php-fpm:phalcon
        ports:
        - "80:80"
        restart: always
        environment:
          WEB_DB_HOST: db:3306
          WEB_DB_PASSWORD: root
        volumes:
        - ./web/html:/var/www/html/
        links:
        - db
    

    这里的environment定义了环境变量,比如host的MAC,就可以在docker容器的系统变量里面得到,各个编程语言都有获取系统环境变量的方法。这些变量也可以组织起来放在文件里面加载进去,参考这里
    这里的volumes定义了要映射进容器的文件或文件夹或者数据容器,可以参考这里。注意,多个容器共享同一个目录,会出现写冲突,比如MySQL,多个实例,数据目录需要分开。所以设计好你的程序,哪一部分是需要只读的(可以共享),哪一部分是需要写的;写的部分是否可以放到各自临时目录,或者其他公共服务里面。
    运行并查看

    root@thinkpad:/home/compose-web# docker-compose up -d
    Creating network "composeweb_default" with the default driver
    Creating composeweb_db_1
    Creating composeweb_web_1
    root@thinkpad:/home/compose-web# docker-compose ps
          Name                   Command             State              Ports            
    ------------------------------------------------------------------------------------
    composeweb_db_1    docker-entrypoint.sh mysqld   Up      0.0.0.0:3306->3306/tcp      
    composeweb_web_1   /start.sh                     Up      443/tcp, 0.0.0.0:80->80/tcp 
    #也可以使用原来的命令
    root@thinkpad:/home/compose-web# docker ps
    CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                         NAMES
    efbdaf257748        nginx-php-fpm:phalcon   "/start.sh"              13 seconds ago      Up 11 seconds       0.0.0.0:80->80/tcp, 443/tcp   composeweb_web_1
    a6935d20911e        mysql:5.7               "docker-entrypoint.sh"   14 seconds ago      Up 13 seconds       0.0.0.0:3306->3306/tcp        composeweb_db_1
    
    

    docker-compose up -d ,将会在后台启动并运行所有的容器。
    docker-compose仅仅提供了对多个docker服务的管理,仍然可以在这些容器上运行原来的docker命令。检查下web容器的IP:

    root@thinkpad:/home/compose-web# docker inspect composeweb_web_1 | grep IPAddress
                "SecondaryIPAddresses": null,
                "IPAddress": "",
                        "IPAddress": "172.18.0.3",
    

    然后访问:http://172.18.0.3/ 就可以看到web页面了。
    如果想要停止服务,可以用docker-compose stop:

    root@thinkpad:/home/compose-web# docker-compose stop db
    Stopping composeweb_db_1 ... done
    root@thinkpad:/home/compose-web# docker-compose stop web
    Stopping composeweb_web_1 ... done
    

    再次启动,得到相同名称的服务:

    root@thinkpad:/home/compose-web# docker-compose up -d
    Starting composeweb_db_1
    Starting composeweb_web_1
    

    各个容器的名称也可以在配置文件里面通过参数container_name来指定。
    docker-compose的配置还有很多其他的参数,可以参考这里。比如刚才我们通过docker inspect来查找容器IP,也可以配置成静态IP:

    version: '2'
    
    services:
      app:
        image: nginx-php-fpm:phalcon
        networks:
          app_net:
            ipv4_address: 172.18.0.10
            ipv6_address: 2001:3984:3989::10
    
    networks:
      app_net:
        driver: bridge
        driver_opts:
          com.docker.network.enable_ipv6: "true"
        ipam:
          driver: default
          config:
          - subnet: 172.18.0.0/24
            gateway: 172.18.0.1
          - subnet: 2001:3984:3989::/64
            gateway: 2001:3984:3989::1
    

    也可以自定义网络
    刚才我们的配置文件里面直接指定了镜像,也可以指定Dockerfile镜像构建,假如Dockerfile在web目录下面:

    build:
      context: ./web
    

    或者直接指明Dockerfile:

    build:
      context: .
      dockerfile: Dockerfile-alternate
    

    实际上,docker-compose的配置已经覆盖了Dockerfile的配置项,也可以直接用于构建多个docker服务,比如指定运行的命令

    command: [/bin/bash]
    

    这里的入口程序,如果不能持续运行的话,运行完成后,docker就会退出。所以如果你需要可以后台运行的docker容器,那么入口程序,就必须保持一个程序在前台运行,不退出,比如运行crontab的镜像,入口程序可以是:

    cron && bash
    

    cron这个程序是在后台运行的,如果不配合前台程序bash,容器执行玩cron就会退出了。其他运行crontab的容器,大都是强制一个程序在前台不退出,比如这个

    cron && tail -f /var/log/cron.log
    

    这里需要注意一下,通过docker设置的environment values在容器内的cli下面执行时取不到的,比如printenv,使用cron后台运行时取不到外部设置的变量,但是手动执行时又可以。这些外部环境变量只能在ENTRYPOINT或者CMD所执行的shell里面去设置,比如在CMD执行start.sh:

    printenv | grep -E "^HOST" > /root/env.conf && cron && bash
    

    Docker compose还可以跟Docker Swarm结合。

    参考链接:
    Install Docker Compose
    Quickstart: Docker Compose and WordPress
    YAML 模板文件
    Introduction to Docker Compose Tool for Multi-Container Applications
    Dockerfile基本结构
    How to Get Environment Variables Passed Through docker-compose to the Containers
    Access environment variables from crontab into a docker container
    How can I access Docker set Environment Variables From a Cron Job
    Dockerfile里指定执行命令用ENTRYPOING和用CMD有何不同?
    What is the difference between CMD and ENTRYPOINT in a Dockerfile?
    Docker difference between run, cmd, entrypoint commands

    基于Docker的Nginx + PHP-FPM + Phalcon镜像

    上一篇简单介绍了Docker的安装,运行,这一篇来构建一个基于Nginx和PHP-FPM的Phalcon镜像。在官方找了以下,单独的Nginx和PHP镜像更加流行,混合的反倒不是很受欢迎。其实官方并不提倡在一个容器里面运行多个服务,最好是一个容器只对外提供一个服务:一个容器启动时仅仅运行一个命令(其实里面可以包含多个),也方便部署扩展升级。多个服务之间可以使用Docker Compose来管理。但是Docker并不阻止创建包含多个服务器的镜像,为了方便,所以我们仍然可以自己构建。
    构建镜像可以有好几种方式,比如基于Alpine Linuxphusion/baseimage-docker构建,或者基于Ubuntu,CentOS等构建,又或者在PHP,Nginx的基础镜像上构建。注意:如果要采用Ubuntu或者CentOS构建,可能需要一些额外的工作,以便保持镜像轻量稳定运行。
    这里采用已有的richarvey/nginx-php-fpm来构建,它是一个基于Nginx官方镜像来构建的。
    Github上拉取相关文件从Dockerfile构建:

    $ sudo git clone https://github.com/ngineered/nginx-php-fpm
    $ sudo docker build -t nginx-php-fpm:latest .
    

    关于Dockerfile的相关解释,可以参考这里。当然也可以直接拉取镜像使用

    $ sudo docker pull richarvey/nginx-php-fpm
    # 也可以直接运行,会自动拉取
    #$ sudo docker run -d richarvey/nginx-php-fpm
    

    查看本地的镜像,连单独的nginx也来了:

    root@thinkpad:~# docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    nginx-php-fpm       latest              4fc9ac9f2945        7 hours ago         228.5 MB
    nginx               mainline-alpine     00bc1e841a8f        5 days ago          54.21 MB
    

    这里的mainline-alpine是指基于Alpine Linux构建的。Alpine Linux是一个仅有5M大小的linux系统,采用apk add/search来安装/查找相应软件,有许多镜像都是基于它构建的,官方PHP镜像也有基于它构建的Docker镜像。
    然后运行nginx-php-fpm:

    root@thinkpad:~# docker run --name web -d richarvey/nginx-php-fpm
    

    docker inspect命令用来查看容器的相关信息,查看下分配的IP:

    root@thinkpad:~# docker inspect web | grep IPAddress
                "SecondaryIPAddresses": null,
                "IPAddress": "172.17.0.2",
                        "IPAddress": "172.17.0.2",
    

    然后在浏览器里面访问:http://172.17.0.2/就可以看到phpinfo的页面。到这里,Nginx + PHP的web容器就已经运行起来了,对应的Nginx和PHP进程可以在宿主机器上直接查看:

    root@thinkpad:~# ps aux | grep nginx
    root     18167  0.0  0.0  13696  4300 pts/6    S    01:47   0:00 nginx: master process /usr/sbin/nginx
    systemd+ 18168  0.0  0.0  14144  1868 pts/6    S    01:47   0:00 nginx: worker process
    systemd+ 18169  0.0  0.0  14144  1868 pts/6    S    01:47   0:00 nginx: worker process
    systemd+ 18170  0.0  0.0  14144  1868 pts/6    S    01:47   0:00 nginx: worker process
    systemd+ 18171  0.0  0.0  14144  1868 pts/6    S    01:47   0:00 nginx: worker process
    systemd+ 18172  0.0  0.0  14144  1868 pts/6    S    01:47   0:00 nginx: worker process
    root     18190  0.0  0.0  21292  1012 pts/18   S+   01:47   0:00 grep --color=auto nginx
    root@thinkpad:~# ps aux | grep php-fpm
    root     18166  0.0  0.2 167880 23364 pts/6    S    01:47   0:00 php-fpm: master process (/etc/php5/php-fpm.conf)
    systemd+ 18173  0.0  0.1 167880  8620 pts/6    S    01:47   0:00 php-fpm: pool www
    systemd+ 18174  0.0  0.1 167880  8620 pts/6    S    01:47   0:00 php-fpm: pool www
    systemd+ 18175  0.0  0.1 167880  8620 pts/6    S    01:47   0:00 php-fpm: pool www
    root     18192  0.0  0.0  21292  1032 pts/18   S+   01:47   0:00 grep --color=auto php-fpm
    

    接下来要为这个容器添加Phalcon扩展。首先要进入容器里面,使用docker attach命令进入:

    root@thinkpad:~# docker attach web
    
    
    
    

    结果在这里等了半天进不去。。。。查看下当前镜像入口程序:

    root@thinkpad:~# docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    94176348a939        nginx-php-fpm       "/start.sh"         6 seconds ago       Up 5 seconds        80/tcp, 443/tcp     web
    

    这个容器启动的时候运行的是start.sh这个脚本,这个脚本运行了Supervisor工具。于是重新启动容器,运行/bin/bash

    #终止容器运行
    root@thinkpad:~# docker stop web
    web
    #删除容器
    root@thinkpad:~# docker rm web
    web
    #重新运行
    root@thinkpad:~# docker run --name web -d -t -i nginx-php-fpm /bin/bash
    ea21e10df702644a83ed75930b30c7764a786c4feabdf17cd868f86640137c47
    root@thinkpad:~# docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    ea21e10df702        nginx-php-fpm       "/bin/bash"         6 seconds ago       Up 5 seconds        80/tcp, 443/tcp     web
    root@thinkpad:~# docker attach web
    #进来了
    bash-4.3# ls
    bin       etc       lib       media     proc      run       srv       sys       usr
    dev       home      linuxrc   mnt       root      sbin      start.sh  tmp       var
    

    就可以进去了。
    先安装编译相关工具包:

    bash-4.3# apk --no-cache add php5-dev
    bash-4.3# apk --no-cache add gcc
    bash-4.3# apk --no-cache add make
    bash-4.3# apk --no-cache add autoconf
    bash-4.3# apk --no-cache add libc-dev
    fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gz
    fetch http://dl-cdn.alpinelinux.org/alpine/v3.4/community/x86_64/APKINDEX.tar.gz
    (1/2) Installing musl-dev (1.1.14-r12)
    (2/2) Installing libc-dev (0.7-r0)
    OK: 334 MiB in 106 packages
    

    编译安装Phalcon:

    bash-4.3# cd /home
    bash-4.3# git clone --depth=1 git://github.com/phalcon/cphalcon.git
    bash-4.3# cd cphalcon/build
    bash-4.3# ./install
    bash-4.3# ls -la /usr/lib/php5/modules/ | grep phalcon
    -rwxr-xr-x    1 root     root       5045264 Sep 28 17:34 phalcon.so
    

    更改PHP扩展的配置:

    bash-4.3# cd /etc/php5/conf.d/
    bash-4.3# vi phalcon.ini
    #添加以下内容
    #extension=phalcon.so
    
    #检查扩展是否加载成功
    bash-4.3# php -i | grep phalcon
    /etc/php5/conf.d/phalcon.ini,
    phalcon
    phalcon => enabled
    phalcon.db.escape_identifiers => On => On
    phalcon.db.force_casting => Off => Off
    phalcon.orm.cast_on_hydrate => Off => Off
    phalcon.orm.column_renaming => On => On
    phalcon.orm.enable_implicit_joins => On => On
    phalcon.orm.enable_literals => On => On
    phalcon.orm.events => On => On
    phalcon.orm.exception_on_failed_save => Off => Off
    phalcon.orm.ignore_unknown_columns => Off => Off
    phalcon.orm.late_state_binding => Off => Off
    phalcon.orm.not_null_validations => On => On
    phalcon.orm.virtual_foreign_keys => On => On
    OLDPWD => /home/cphalcon/build
    _SERVER["OLDPWD"] => /home/cphalcon/build
    _ENV["OLDPWD"] => /home/cphalcon/build
    

    加载成功了,需要保持本次镜像变更。首先退出容器:

    bash-4.3# cd /home
    #删除各种不必要的东西,比如gcc
    bash-4.3# rm -rf cphalcon/
    bash-4.3# exit
    exit
    

    然后查看版本并提交变更:

    root@thinkpad:~# docker ps -l
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
    ea21e10df702        nginx-php-fpm       "/bin/bash"         31 minutes ago      Exited (0) 6 seconds ago                       web
    root@thinkpad:~# docker commit ea2 nginx-php-fpm:phalcon
    sha256:bb388df328ecc33fac02dba69759d5c992a145f650a0e5b20ca29a4b122fa933
    

    docker commit命令可以用来提交变更,ea2是container id的前三位,也可以写全;然后跟的是要提交的镜像。这里提交到phalcon这个标签下,以便与原来的区分开。查看所有镜像,发现有两个不同的标签:

    root@thinkpad:~# docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    nginx-php-fpm       phalcon             bb388df328ec        11 seconds ago      364.4 MB
    nginx-php-fpm       latest              4fc9ac9f2945        4 hours ago         228.5 MB
    

    采用新镜像来运行,这里要讲程序运行入口改回/start.sh,以便能正常启动Nginx和PHP-FPM:

    root@thinkpad:~# docker rm web
    web
    root@thinkpad:~# docker run --name web -d -t -i nginx-php-fpm:phalcon /start.sh
    deecb19467cda2676b24248e3f55970a2481255c6022a80ffbf5087792ccb559
    root@thinkpad:~# docker ps
    CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS              PORTS               NAMES
    deecb19467cd        nginx-php-fpm:phalcon   "/start.sh"         4 seconds ago       Up 3 seconds        80/tcp, 443/tcp     web
    

    入口程序改变了,需要再提交一次变更:

    root@thinkpad:~# docker stop web
    web
    root@thinkpad:~# docker ps -l
    CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS                       PORTS               NAMES
    c7600e62733d        nginx-php-fpm:phalcon   "/start.sh"         34 seconds ago      Exited (137) 8 seconds ago                       web
    root@thinkpad:~# docker commit c76 nginx-php-fpm:phalcon
    sha256:1c97ee169a551dd8441f42b40beafd102c71f3e887e2317dc11ce0ef136ceaf0
    

    运行最终的镜像:

    root@thinkpad:~# docker rm web
    web
    root@thinkpad:~# docker run --name web -d -t -i nginx-php-fpm:phalcon
    cb5b0c9e55913a538539e46c53ac7905b21def84a05eb00ef81c4b500853576c
    root@thinkpad:~# docker ps
    CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS              PORTS               NAMES
    cb5b0c9e5591        nginx-php-fpm:phalcon   "/start.sh"         4 seconds ago       Up 3 seconds        80/tcp, 443/tcp     web
    

    访问http://172.17.0.2/,便可以在页面找到phalcon扩展。
    通常我们会将程序和数据分开,挂载外部文件目录到容器里面去:

    root@thinkpad:~# docker stop web
    web
    root@thinkpad:~# docker rm web
    web
    root@thinkpad:~# docker run --name web -d -t -i -v /home/docker/nginx-php-fpm/src:/var/www/html/ nginx-php-fpm:phalcon
    ffd64793fe8e7a2a95b68f514e221b7ec3b6cadfe668c016f55a7bb6d48bc702
    

    -v参数可以用来挂载目录或者文件,可以又多个-v参数。
    刚才容器里面做的那些已经添加到Dockerfile里面去,你直接使用它来构建。
    至此Nginx + PHP-FPM + Phalcon镜像构建完成,介绍绍了如何进入容器,提交变更,网络访问和文件挂载。

    参考链接:
    A minimal Ubuntu base image modified for Docker-friendliness
    eboraas/phalcon
    基于Docker的PHP开发环境
    Docker for PHP Developers
    Docker在PHP项目开发环境中的应用
    使用 Supervisor 来管理进程
    PHP C扩展框架Phalcon
    Alpine Linux,一个只有5M的Docker镜像