作者：hqiang1984

量化自我，极简主义

Pylearn2的使用简介

环境: ubuntu 12.4

Pylearn2是基于theano上封装的深度学习包。 它实现一些常见的模型，具体请参考： http://deeplearning.net/software/pylearn2/library/index.html#libdoc，比theano在做实际的项目节约时间，只需要配置一些参数来实现模型的训练。
下面来讲解实际的安装和使用：

1. 安装Theano（Bleeding-edge install instruction）

    jerry@hq:~$sudo pip install –upgrade –no-deps git+git://github.com/Theano/Theano.git –user

2. 下载Pylearn2
jerry@hq:~$git clone git://github.com/lisa-lab/pylearn2.git

3.  安装pylearn2
jerry@hq:~$cd pylearn2
jerry@hq:~$sudo python setup.py develop –user

4. 测试安装成功
jerry@hq:~$python
import pylearn2
能加载包即安装ok

5. 设置PYTHON2_DATA_PATH, PYLEARN2_VIEWR_COMMAND
vi ~/.bashrc
添加
export PYLEARN2_DATA_PATH=/u01/lisa/data
export PYLEARN2_VIEWER_COMMAND=/usr/bin/eog

如何运行一个示例

1. 下载数据
cd /u01/lisa/data/cifar10
wget http://www.cs.utoronto.ca/~kriz/cifar-10-python.tar.gz
tar xvf cifar-10-python.tar.gz

2. 修改make_dataset.py文件 ，指定路径/u01/lisa/data/ (由于本机上/空间不足，只能把数据放在其它路径上)
jerry@hq:~$vi /home/jerry/pylearn2/pylearn2/scripts/tutorials/grbm_smd/make_dataset.py
修改成这样：
“””
path = pylearn2.__path__[0]
train_example_path = os.path.join(path, ‘scripts’, ‘tutorials’, ‘grbm_smd’)
train.use_design_loc(os.path.join(train_example_path, ‘cifar10_preprocessed_train_design.npy’))
train_pkl_path = os.path.join(train_example_path, ‘cifar10_preprocessed_train.pkl’)
“””
train_pkl_path = os.path.join(‘/u01/lisa/data/’, ‘cifar10_preprocessed_train.pkl’)
serial.save(train_pkl_path, train)

3. 对下载数据进行数据预处理
python /home/jerry/pylearn2/pylearn2/scripts/tutorials/grbm_smd/make_dataset.py
处理完后在目录/u01/lisa/data下有一个文件 cifar10_preprocessed_train.pkl，大概652M左右

4. 对数据进行训练
cd /u01/lisa/data
python ~/pylearn2/pylearn2/scripts/train.py ~/pylearn2/pylearn2/scripts/tutorials/grbm_smd/cifar_grbm_smd.yaml

5. 查看结果
python ~/pylearn2/pylearn2/scripts/show_weights.py ~/pylearn2/pylearn2/scripts/tutorials/grbm_smd/cifar_grbm_smd.pkl

python ~/pylearn2/pylearn2/scripts/plot_monitor.py ~/pylearn2/pylearn2/scripts/tutorials/grbm_smd/cifar_grbm_smd.pkl

python ~/pylearn2/pylearn2/scripts/print_monitor.py ~/pylearn2/pylearn2/scripts/tutorials/grbm_smd/cifar_grbm_smd.pkl

    python ~/pylearn2/pylearn2/scripts/summarize_model.py ~/pylearn2/pylearn2/scripts/tutorials/grbm_smd/cifar_grbm_smd.pkl

 

6. 直接查看生成参数的文件cifar_grbm_smd.pkl

加载模型文件
>>> from pylearn2.utils import serial
>>> model = serial.load(‘/home/jerry/pylearn2/pylearn2/scripts/tutorials/grbm_smd/cifar_grbm_smd.pkl’)
查下文件结构
>>> dir(model)
获取权重参数
>>> model.get_weights()
获取参数名
>>> model.get_params()
获取参数值
>>> model.get_param_values()

python 邮件发送

python 2.7代码如下：

#coding: utf-8
import smtplib
from email.mime.text import MIMEText

#connect smtp server
msg = MIMEText(‘Hello’,’plain’,’utf-8′)
msg[‘Subject’] = ‘ Load data sucess!’
#msg[‘Date’] = formatdate(localtime=True)
smtp = smtplib.SMTP()
smtp.connect(‘proxy-in.xxx.com’)
smtp.sendmail(‘bidev@xxx.com’, ‘swer@xxx.com’, msg.as_string())

 

python 2.4.3

#coding: utf-8
import smtplib
from email.MIMEText import MIMEText

#connect smtp server
msg = MIMEText(‘Hello’,’plain’,’utf-8′)
msg[‘Subject’] = ‘ Load data sucess!’
smtp = smtplib.SMTP()
smtp.connect(‘proxy-in.xxx.com’)
smtp.sendmail(‘bidev@xxx.com’, ‘hsdf@xxx.com’, msg.as_string())

http://m.baidu.com/news?fr=mohome&ssid=0&uid=&pu=sz%401320_2001%2Cta%40iphone_1_7.1_3_537&bd_page_type=1#page/info%3A互联网/http%3A%2F%2Fwww.huxiu.com%2Farticle%2F114327%2F1.html/深挖BAT内部级别和薪资待遇，你敢看？%20/虎嗅网/1430894729000/12489789468531375681

CXXNET安装

环境：ubuntu 14.04,  cuda 6.5

先安装cuda-toolkit, cuda-cublas, cudart, cuda-curand这四个安装包

cuda_6.5.14_linux_64.run

cuda-cublas-6-5_6.5-14_amd64.deb
cuda-cudart-6-5_6.5-14_amd64.deb
cuda-curand-6-5_6.5-14_amd64.deb

下载路径：http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/

安装 OpenCV

sudo apt-get install libopencv-2.4

 

配置环境变量

vi ~/.bashrc

export CUDA_HOME=/usr/local/cuda-6.5
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:/usr/local/lib:$LD_LIBRARY_PATH
export CPLUS_INCLUDE_PATH=/usr/local/cuda/include

 

下载一份cxxnet

git clone https://github.com/dmlc/cxxnet.git

切换至目录 cd cxxnet

拷贝一份配置到当前目录 cp make/config.mk .

修改 vi config.mk

USE_CUDA = 1

USE_BLAS = blas

USE_DIST_PS = 1
USE_OPENMP_ITER = 1

编辑 vi  Makefile， 修改如下：

CFLAGS += -g -O3 -I./mshadow/  -fPIC $(MSHADOW_CFLAGS) -fopenmp -I/usr/local/cuda/include
LDFLAGS = -pthread $(MSHADOW_LDFLAGS) -L/usr/local/cuda/lib64

 

最后编译文件

./build.sh

 

 

unable to correct problems you have held broken packages

OS:  Ubuntu 14.04

When installing a ubuntu desktop KDE,  some error  like “unable to correct problems you have held broken packages” happen.  So I finally found the problm is the package python3-software-properties is too new and can’t be compatible with the kde package. The soluation is follling:

sudo apt-get remove python3-software-properties

sudo apt-get install python3-software-properties=0.92.36

在百度的最后一天

今天(2015-03-13)是办理离职流程的最后一天。归还公司的资产，到财务结清工资并拿离职证明。过程还比较顺利。感觉有些轻松又有些伤感。从2012年9月入职到现在离开，时间过的可真快。有成长，有郁闷，有辛酸，现在真有些不是滋味。或许以后还会再回来，不知道。人生有几个三年，来回折腾人生。有些老了，或许不够壮志，但路要走得稳。希望自己在新单位发展如意！