All I need is time.
基于OpenCV自动检测物体并将背景透明
2012年12月26日
2 条评论
这个项目结束整整两个月了，一直想把那段时间的付出拿来跟别人分享一下，也算是对自己这学期这个项目的一个总结吧。
这个项目的背景，是博物馆不便将大量的考古物品交给考古学家来研究，而考古学家们也不便满世界飞去进行实地考察。因此，博物馆用一套摄影装置对这些物体拍照，然后将照片送给考古学家，以便让他们进行研究。每个物体需要从各个不同的角度照大量的照片，而背景可能不为纯色，含有其他物体，考古学家们便希望能将背景去掉，只保留有价值的部分——那些考古文物。
下面左边是原图，右边的经过处理后的结果。实际上，对于前三中纯色背景的图，我们的程序可以较好地应对，但是对于最后一个混合背景的图，效果并不是很理想，时常会出现有背景被留了下来，这点有待提高。
在项目刚开始的时候，我们想出了多种方法，比如将前后两张照片进行对比，找出变换了的部分，在比如分析照片的像素，找出背景的颜色等等。但是实现起来比较不容易，效果也不一定好。最后组长Paul提出使用OpenCV的Grabcut，最后证明，Grabcut还是很智能很犀利的。
好了，进入正题。
对于OpenCV的安装和配置就不多说了，我们使用的是OpenCV 2.4.2和VS2010。
首先我们读入图片，并存为OpenCV中的Mat
|
1 |
Mat img = imread(inputimgname); |
为了找出物体在哪里，我们先使用Canny算法将图片中的边缘部分找出来。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
Mat gray = Mat(img.size(), CV_8U); cvtColor(img, gray, CV_BGR2GRAY); Size_<int> blurParam; blurParam.height = 7; blurParam.width = 7; GaussianBlur(gray, gray, blurParam,1.55); Mat canny = Mat(img.size(), CV_8U); Canny(gray, canny, min_thresh, max_thresh, 3);//这里设置最大最小阈值，阈值设得越低，结果的像素越多 |
效果如下图：
但是这点儿边缘的像素点儿够用么？我想还不够。所以我们想把这些像素点放大。最开始我自己写了个算法，暴力得遍历几乎每一个像素，如果它的白的，便把它周围一个范围内的像素都变为白的。但是这样的效率实在的太低了，处理一张图都要几十秒，这可不行。后来搜了一下，OpenCV果然强大，自带这个函数——dilate & erode。其中dilate把白色的像素点进行方法，而erode是把黑色的部分放大。这样除了能把边缘像素放大了之后，还可以把背景上一些噪点给去掉。效果如下图：
恩，看起来不错，代码如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
Mat img; dilate(img,img_src,Mat(Size(dilate_thresh, dilate_thresh),CV_8UC1)); int minarea = minarea_thresh; int tmparea = 0; Mat img_Clone = img_src.clone(); vector<vector<Point> > contours; findContours(img_src, contours, CV_RETR_CCOMP, CV_CHAIN_APPROX_NONE,Point(0,0)); CvRect rect; vector<Point> contour; for(int i =0; i< contours.size(); i++){ contour = contours[i]; tmparea = fabs(contourArea(contour)); rect = boundingRect(contour); if (tmparea < minarea/*||tmparea>4900*/) { //pp=(uchar*)(img_Clone.data + img_Clone.widthStep*(rect.y+rect.height/2)+rect.x+rect.width/2); unsigned char pp = img_Clone.at<unsigned char>(rect.y+rect.height/2, rect.x+rect.wi dth/2); if (pp == 255) { for(int y = rect.y;y<rect.y+rect.height;y++) { for(int x =rect.x;x<rect.x+rect.width;x++) { pp=img_Clone.at<uchar>(y,x); if (pp == 255) { pp = 0; } } } } } } Mat better(img_Clone); Mat eroded_mask(better.rows,better.cols,CV_8UC1); for(int r = 0;r < better.rows;r++){ for(int c = 0;c < better.cols;c++){ if(better.at<uchar>(r,c) == 255){ eroded_mask.at<uchar>(r,c) = GC_PR_FGD; }else{ eroded_mask.at<uchar>(r,c) = GC_PR_BGD; } } } return eroded_mask; |
最后eroded_mask就是上图了，它将作为grabcut函数的输入。
下面便是大名鼎鼎的grabcut了！代码如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
Rect rect1(0,0,1,1);//rectangle for internal use Mat bgModel, fgModel; // models for internal use Size2i miniSize; miniSize.width = initial_img.size().width/2; miniSize.height = initial_img.size().height/2; Mat miniImage = Mat(miniSize, CV_8U); Mat miniMask = Mat(miniSize, CV_8U); //pyrDown(mask_feature, miniMask,miniSize); pyrDown(initial_img, miniImage, miniSize); for(int i=0; i<initial_img.rows; i += 2){ for(int j = 0; j<initial_img.cols; j += 2){ miniMask.at<unsigned char>(i/2,j/2) = mask_feature.at<unsigned char>(i,j); } } grabCut(miniImage, miniMask, rect1, bgModel, fgModel, iterate, mode); for(int j=0; j<miniSize.height; j++){ for(int i=0; i<miniSize.width; i++){ mask_feature.at<unsigned char>(2*j,2*i) = miniMask.at<unsigned char>(j,i); mask_feature.at<unsigned char>(2*j+1,2*i) = miniMask.at<unsigned char>(j,i); mask_feature.at<unsigned char>(2*j,2*i+1) = miniMask.at<unsigned char>(j,i); mask_feature.at<unsigned char>(2*j+1,2*i+1) = miniMask.at<unsigned char>(j,i); } } compare(mask_feature, GC_PR_FGD, mask_feature, CMP_EQ); mask_feature = mask_feature & 1; return mask_feature; |
中间Paul加了点儿处理，将用到的图的长宽都缩小一倍，进行处理，最后再返回原大小，这样可以显著提高效率，且对结果没有太大的影响，我想是因为原图的大小足够大吧。最后返回的mask_feature标记出了哪里是图片中的前景，也就是物体，其余部分为背景。然后根据这个返回的mask将所有背景像素透明，并存为png格式的图片。透明部分的代码如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
Mat final = Mat(img.size(), CV_8UC4);//img为原图 for ( int nrow = 0; nrow < img.rows; nrow++) { for (int ncol = 0; ncol < img.cols; ncol++){ if (mask.at<uchar>(nrow,ncol) == GC_BGD ){//mask为grabcut返回的图，根据它的像素值来进行透明处理。如果是GC_BGD，也就是background时，就将这个像素透明化 for(int i=0;i<4;i++){ final.at<Vec4b>(nrow,ncol)[i] = 0; } }else{ for(int i=0; i<3; i++){ final.at<Vec4b>(nrow,ncol)[i] = img.at<Vec3b>(nrow, ncol)[i]; } final.at<Vec4b>(nrow, ncol)[3] = 255; } } } return final;//返回最终的Mat图 最终只需要保存图片就好了 imwrite(imageName,final); |
当然，对于整个项目来说，还需要有很多其他的步骤，比如连续读入图片，然后根据要求生成保存图片的名字，判断用户输入是否符合要求等。但对于图像处理的部分主要就是这些。现在想想，其实并没有那么复杂，代码也不是很多，最终的结果也许并不是那么完美，还有很多地方需要提高，但在做这个项目的过程中我们也确实学到了不少，谢谢给力的队友Patrick，当然还有Paul，明年的project不一定在一组了，但都继续加油吧！
再次看到天舒的信
2012年11月1日
没有评论
今儿再次打开了QQ邮箱，再次看到了天舒给我的信，仿佛回到了跟你跟大家在一起踢球的日子，真好。上次你说球队现在在努力训练，大家进步也不小，我很欣慰。我多么希望明年能回去看你们夺冠啊！加油！为了电子！
=====================================================
致华烨：
首先，谢谢你送的足球，我想它对于我，甚至整个球队都有重要的意义。我想它会保佑我们，见证我们创造一个新的电子王朝。
上次吃饭说有东西要送给你，显然，不管我怎么精心挑选我送的礼物也不会比你亲手制作的足球更有意义。所以我选择写信，一方面，写信是传达情感的最好方式；另一方面，我的确有很多的话想说。
我们有共同的爱好和性格。我认为这是我们惺惺相惜的地方。很多时候，我认为我们能互相理解对方。即便如此，我想我还是不太可能比较完整的了解你，毕竟我们不是一个年级。平时除了踢球相处的时间很少。但就是这很少的时间里，我认为我们的关系是愉快轻松加融洽的。
我个人觉得，嗯，你适合踢后腰，第一次和你踢球的时候你的控球给了我深刻的印象。我觉得你接球，护球，控球，然后传球拿下对于全队来说是很关键的。实话说，当边锋你的突破不够犀利啊，所以，如果你去澳洲还有机会踢球的话，可以考虑回到后腰的位子。对了，还要多练任意球啊……你看校长杯的任意球大部分让你罚了，结果还没进个球。。。
实话说，你把妹的理论知识很是丰富啊，可是为什么就把不到呢？虽然我也没资格说你，但是还是很不解啊。我们的“华哥”这么受欢迎竟然连个妹子都没有，岂不让人笑话？不过，话又说回来，一个人也挺好，俗话说得好，好女费汉，我们级的于斌，徐尧，现在的段一飞就是例证。。。不过这都是玩笑话啦，还是希望你早点找到陪你看星星的那个人，嗯。
11级的实力相信你也清楚，还是很不错的。我想等他们来之后，我们的阵容可以做相应的变化。可以加强中场的控制（他们有几个控球和传球都不错）。但是比较棘手的问题是门将和中后卫。俊鹏和思远的差距还很大。而且我们队现在也没有一个像马小龙那样的后场核心。这都是后面一年要解决的问题。我相信明年的校长杯我们一定能拿到冠军的，到时候如果你能来看到然最好，如果不能，我会把照片寄过去的。
听说澳洲风景不错，在那边读书应该比较惬意吧。既然你说那边会很少踢球，那就好好学啊。你要相信，学术大牛总是会找到好妹子的，这是亘古不变的真理。
认识两年来，其实严格算也就大二一年，深受你和大四其他学长的照顾，不管是在足球还是生活方面。十分感谢。
最后，希望你能在澳洲那边混得很好吧。
你真诚的朋友,
王天舒
P.S:乱说了一大堆，不要介意。自从高中毕业后，写作能力直线下降，所以你领会大意就行。。。
我唱自己的歌
在布满车前草的道路上
在灌木的集市上
在雪松和白桦树的舞会上
在那山野的原始欢乐上
我唱自己的歌
……
回家？
2012年10月28日
1 条评论
此刻，我坐在空无一人的workroom，放着音乐，喝着喜欢的香蕉牛奶，作业做完了，又不想复习。难得清闲，写点儿什么吧。
明儿早上要去找客户Dirk，这个追求完美的死老头儿，一定要让他接受我们做的东西。
昨儿是育鸿50周年校庆，看到很多原来的同学回去了，很是羡慕他们，甚至晚上梦到了大家又聚到了一起，不过居然是在墨尔本 = = 。老妈你的照片呢？快给我看看啊~
也许是期末了压力大的原因，本来一直没有打算这个假期回家的，最近越发地想回去了。见父母，见亲人，见同学，见朋友，吃好吃的，过年，等等等等。是啊，回去固然好处不少。也许哪天一激动就订票了呢。
第一学期的课已经上完了，7号就要开始考试了，好好复习，迎接考试的到来吧！
Individual Report of Project
2012年10月26日
3 条评论
My name: Ye Hua
- write the introduction part of SRS
- convert part of other's word file of SRS to LaTeX file (Paul did same task)
- convert part of other's word file of SAD to LaTeX file (Paul did same task)
- convert TP to LaTeX file
- SAD internal review
- As a client liaison, meet with Dirk every Monday and made notes of his requirements
- regularly update the wiki page, including the Project Plan, Meeting with client, Tasks Table, internal reviews, and store the emails from client on it
- update the repository and commit my work after I finish it
- tell team members how to use SVN
- send email to the team address to suggest that we have to have regular meeting instead of working separately
- write some java code to test canny algorithm first and use it as demo
- Implementation: I spent every day working on the project in the last 3 weeks, stay in the McDonell 5.06 till 1am every night. I implemented the canny feature detection, dilate and erode, rotation, and copy files to the destination directory parts of our program. I also helped Patrick to implemented the transparent and conversion to png, make new directories and save images with right name parts. I wrote the manpage of our program. Create the on-line help page. Implement renaming part with Patrick. Fix the bug that the input directories must end up with '/'
- test the program on my own laptop, and test the program on the server after it's integrated.
- update the test log on the repository
- attend almost every meeting, including team meeting, client meeting and supervisor meeting (from my memory, I missed only one supervisor meeting). attend all the workshop and lecture. participated in the team discussion. took photos of the notes of the meeting or workshop
- help other team members to work together well
- work with Patrick and Paul in the last 3 weeks in the McDonell to finish the project
- point out the missing stuff that project manager forgot
Team member name 1: Paul
- took notes during the first several meetings, including the client meeting and supervisor meeting
- created the wiki page, including the Minutes, Team roles, and Resources. Updated the wiki page regularly
- do some research in the image processing area, including the format of jpg and png file, OpenCV. find some useful function in OpenCV that may solve our problem
- write most part of the SRS,SAD,SDD, internal review of SAD, audit response
- attend almost every client meeting, clarify the requirements from client and explain them to the team
- as the project manager, arrange the meetings and allocate the tasks to the team members (list the tasks on the wiki or send them via email, and others chose their tasks)
- design the modules of our system, decide that using C++ and OpenCV (even if none of us had much experience in C++…)
- covert some doc files to LaTeX file
- install up to date gcc and OpenCV on the server, which are really important steps in our project
- help Patrick and me to fix bugs in our program, and improve our code
- implemented the reading options part of our program
- keep working during the whole semester, especially the last 2 weeks, Paul slept in the Uni for more than a week
- help Patrick and me to compile and test the program both on our laptops and on the server
- modify the SRS and SDD during the semester
- attend every meeting and workshop and lecture
Team member name 2: Patrick
- attend most of the meetings, workshops and lectures, participated in the group discussion
- write part of the SRS,SAD and TP
- write internal review of SRS
- write audit
- as a risk manager, keep updating the risk log on the wiki page
- try some matlab code to find a way to solve our project
- study C++ and OpenCV during the mid-break
- teach me how to use VS2010 and OpenCV, teach me some C++ stuff
- implementation: write the code of reading in the image from source directory, converting image from jpg to Mat in OpenCV, removing the noisy pixels in the mask, trasparenting the background the save the image as png file in the destination directory.Implement renaming part with me.
- test the program on his laptop and server during the implementation
- update the wiki page and repository regularly and send email to team members to remind the changes
- spend a lot of time working on the project, especially the last 3 weeks, Patrick,Paul and I were working together to implement the program
- help me to improve my code
Team member name 3: Karlen
- attend some of the team meeting and supervisor meeting, attend the first client meeting (he was the client liaison at first, but he didn't do his part of work well, so he changed the role with me). attend some workshops and participated in the group discussion. never attend the lecture
- write some parts of SRS,SAD,SAD internal review and audit.
- give us the sample code of grabcut
- modify the TP
Team member name 4: Sai
- attend some of the team meeting and supervisor meeting, meet with client twice
- write some parts of the SRS,SAD and TP
- write SRS internal review and audit
- as a software administrator, his did nothing related to his role
- modify TP, writing requirements tracking table
General comments:
First of all, I'm happy to study this subject and work with others as a project team. To be honest, I don't think I can be a project manager in a team and I don't have enough knowledge and technology to finish a project by my own. Apart from that, I also have to improve my english ability to communicate with others better, that's very important when I'm working within a team.
But I realized that our team can not work as a team as I expected before. Paul is a really hardworking guy and concentrated on our project during the semester, but I have to say that maybe he'd better not to be a manager, or it's not the most suitable role for him. Patrick is a good team member, some parts of the reason are I can communicate with him better, you know, but he is hardworking too. He also spent much time on our project and he tought me a lot of technical stuffs. I feel energetic working with him. As for Karlen, he is clever and talkative, but, what he has done is much less than what he has spoke to us. And he always being absent from our meeting without reason. For Sai, I didn't have chance to work with him, but I believe that he made limited contribution to our project, he didn't write a single line of code, even he is software administrator. He was told to install OpenCV on the server, but he said that he can not install it because he didn't have permission. Some days later, Paul installed it successfully. Thanks to Paul, we can continue our work on the server. Although Karlen and Sai didn't come to work in the last few weeks, it seems that they were not afraid of being failed in the subject, I can not understand it. During the last 3 weeks, Patrick, Paul and I spent all of our time working in the workroom 5.06, I enjoyed this period of time as we worked as a team, and finally we finish our project, though it's not perfect. On the 26 Oct, we delivered our program to Dirk, but he was not satisfied with our work because we didn't implement the renaming part. In fact, team 4 has sent us an email saying that he already did this, so we were off the hook. That's the reason why we didn't implement it before. After the client delivery, we fixed some bug in our program, and added this function to our program immediately, and sent an email to Dirk to inform these changes. Hope that we were not too late. You know, we all don't want to fail.
Another thing that makes me unhappy is that, on 26 Oct, Paul told me that there was no chance to pass this subject because the work we have done so far was bad. To be honest, I was really angry at that time because we still had time to finish our project, so we still had chance to pass. Why should we give up? Anyway, we finished our project before last second.
As for my own, I met some difficulties communicating with each other at the begin of this semester due to my english level, but I tried my best to improve it so that I can work with my teammates better. And sometimes I have some ideas I don't know how to express myself. And sometimes I don't know how to complain something, maybe for the decision, maybe for some of my teammates were not working on our project. Anyway, thanks to this subject and this project, I've learned a lot from it, from the technical stuff, to the project management things. Thanks Zoltan and Tom for supervising me, I really appreciate for what you have done for me.
/* fuck! especially Karlen, Sai and Dirk! */
近期评论