This week is a week focused on Video Abstraction. Our research project aims to perform video abstraction and perform some evaluation on how good the abstracted video is. On Monday we had a short discussion with Professor Sun and our Teaching Assistant and narrowed down our focus of this project. Then we worked on the project itself till Friday and subsequently I went to Shanghai for the weekend.
On Monday, we had our weekly Design Fiction class before the short discussion with Professor Sun. The Design Fiction class this week was on poster and magazine cover design. The idea was to convey a story with a single picture or more specifically, to develop photo editing skills to design a picture such that it can convey a message clearly. We had 2 mini projects for this class. One of it was to design a magazine cover using any picture we had taken and the other is to design a poster by combining 2 or more pictures into one, on the topic of “future learning”. Both the mini projects were interesting, and we had a good time making the posters and magazine covers.
After the class, we had a quick discussion with Professor Sun and through the discussion we realized we had to start planning for the research project as only around 4 weeks was left before the final presentation on Week 12. We then broke up into our mini groups where we discussed with the Teaching Assistant on the requirements of the project and how we should scope our ideas this time around. We eventually decided to focus on Live Stream videos as video abstraction is rarely seen done on live streams. We also decided on the topic of Gaming live streaming such as the Twitch platform. Then we brainstormed for the problem that we are going to tackle and we decided upon the problem of live stream viewers feeling lost upon starting the stream. Since I used to watch live streams, I always felt that the start of the live stream felt awkward as it usually starts in the middle of a game that the streamer is playing and I don’t have a good idea on what is going on. Thus, we decided to do video abstraction on the live stream such that the abstracted video can be a good summarization of what went on up till the current moment of the live stream.
For the next 5 days, we brainstormed on what we define to be an ideal video abstract and what is the method that we are going to use to identify these ideal highlights of the game. We thus appointed each member with a few tasks to try out various identification methods such as the volume of the stream, the health of the player or the number of players within the screen. These could give us an identification of whether the current frame is an important one and if it is, it could be transformed into a video skim to be compiled as a video abstract. For me, I tried motion analysis on the video by performing some optical flow identification. This method turned out to be really bad as what the optical flow measures is the background image. In our project, we decided to further narrow down the topic to League of Legends games to help simplify our process and in this game, the player can move the screen around the map and thus move the entire background in the process. Thus, when performing motion analysis, what we measure is how much the background changes and not how much action the in-game character has. Which turns out to be a bad identification method for the purpose of our video abstract.
After working hard on the research project, I went to Shanghai to play for the weekends! This trip is a short trip meant to try out the good food in Shanghai and to explore the various famous spots in Shanghai. I went to Yu Garden, Tian Zi Fang, Xin Shi Jie and Nan Jing Lu while trying some of the good food such as the Huang Yu Noodles and Beef Siew Mai.
Overall, this week was a good week and there was good progress made in the research project and I managed to explore Shanghai.
