Social Network Analysis for Computer Scientists

The main goal of this lab session is to in pairs of teams learn from the other team in terms of how to organize code, experiments and data for the course project.
Ask a course assistant to be assigned to a team upon entering the computer room. We start at 9.00.

What is code review about?

• Peer review: evaluation of work by one or more individuals with similar competence
• Pair programming
• Four eyes see more than two
• Go beyond your current knowledge and skills
• Output: list of best practices

What to do?

• Explain your work to the other team
• Mention what you have done and not yet done
• Introduce the other team to your code
• Ask questions about the other team's code
• Explain to the other team positive and less positive constructive points about their work
• Together, derive useful ``best practices'' and add them to the Google Doc (see website)

Evaluation criteria

• Correctness guarantees
• Time and memory constraints
• Are input and output data validated for consistency?
• Is the output easily reusable for result tables or diagrams?
• Is there a pipeline of experiments to run different algorithms on different datasets?
• Is the data relevant and sufficient?
• Is the data ``diverse'' in relevant dimensions?
• What do you measure in each experiment? \\ Quality, running time, error?
• Why is this data good for these experiments?
• Is the data possibly biased and how may this affect the experiments?

Done? Contribute your findings to the list of best practices

The main goal of this lab session is to get started with the course project paper.

Course project
During the lab session three weeks ago, you have made a project planning. One upcoming deadline is the first version of your paper for the peer review session.

1. Template. Download the paper template and make sure you can compile it and have a way of collaborating on the paper with your team partner.
2. Related work. If you have not yet done so, spend some time at Google Scholar investigating a) what other papers exist on this topic, b) which relevant papers cite your paper, and c) what important references are presented in your paper. You will likely want to include some of these in your paper as well.
3. Project contribution. For the course project, you have to do something original. Ideally, this goes beyond the one paper that you were assigned, comparing techniques from multiple papers, for example comparing different algorithms or methods, using different validation metrics, or testing on (a) larger (number of diverse) datasets. Try to get this written down in the appropriate section(s) of your paper.
4. Writing. Make serious progress with writing the first 3 to 4 sections of your course project paper (also work on this in the coming two weeks).

Assignment 2
Take the opportunity to ask final questions about Assignment 2.

The main goal of this lab session is to get started with both the course project and with Assignment 2, and to get to know the data science lab.

Data Science Lab
The data science lab website provides necessary information and documentation. Become familiar with the lab and how to run code on it, and how to place data within the lab. Remember: your homedirectory is for your own code, /local is for local storage on the current machine you are are on, and /data is for storing data across data science lab machines. You may want to use the lab's facilities for course assignments and your course project.

Course project
Below are some topics you can discuss and investigate together with your project team partner.

1. Project schedule. Have a look at the deadlines for the course project in the schedule on this website, and create a sensible planning with your team partner.
2. Paper. Read your course project paper, and spend some time at Google Scholar investigating a) what other papers exist on this topic, b) which relevant papers cite your paper, and c) what important references are presented in your paper.
3. Data repositories. Check out the data repositories of real (social) networks such as SNAP and KONECT and think of datasets suitable for your course project.
4. Project contribution. For the course project, you have to do something original. Ideally, this goes beyond the one paper that you were assigned, comparing techniques from multiple papers, for example comparing different algorithms or methods, using different validation metrics, or testing on (a) larger (number of diverse) datasets. Write down in at most 300 words what you plan to do for your course project, and discuss this with the lecturer or an assistant for feedback.

Assignment 2
Get started with the practical part of Assignment 2.
Note that the large Twitter dataset are also available in the data science lab in the folder /data/SNACS/.

We will again use the UNIX/Linux workstations in Snellius room 306/304 for this lab session.
There is again no deliverable for this lab session.
The main goal of this lab session is to become familiar with NetworkX (a Python package to analyze networks for research purposes).
All relevant information on NetworkX can be found in the NetworkX online documentation.

Python instructions to use NetworkX: NetworkX should be installed on all machines.
Unfortunately, the past years have learned us that machines sometimes exhibit diversity in consistency in package availability on these machines. In some years, instructions to get NetworkX running could be found in /vol/share/software/datascience/2016-README.
If you run into issues, it may be productive to forget about preinstalled packages and use Anaconda and install required packages yourself.

To use Python in your own environment:
wget http://repo.continuum.io/archive/Anaconda3-4.3.0-Linux-x86_64.sh
bash Anaconda3-4.3.0-Linux-x86_64.sh
Accept questions with 'yes'.
source anaconda3/bin/activate
You should now have your own environment in which you can install packages as you wish.

1. Take some time to do the NetworkX tutorial.
2. Have a look at functionality to read and write graphs from/to disk.
3. A lot of the common network metrics you may want to compute are implemented as NetworkX function. Become familiar with these.
4. Try to load the network from the first lab session (/vol/share/groups/liacs/scratch/SNACS/small-gephiready.tsv) into NetworkX, and investigate some characteristic properties of the network, such as the degree distribution. Use the read_edgelist function.
5. Get the Epinions network from the SNAP repository. Compute some common characteristics, and visualize them using appropriate distributions.
6. What are the differences between NetworkX and Gephi in terms of visualization and analysis capabilities?
7. Proceed with Exercise 2 of Assignment 1.

Looking for a challenge? Check out Graph-Tool, a python graph analysis toolkit that leaves the hard computation to parallel (OpenMP) C++ code. Or snap, which is entirely written in C++ (although there is now also a Python plugin).

We will use the UNIX/Linux Ubuntu workstations in the Snellius computer rooms for this lab session.
This ICT FAQ may provide links to more information on how to use these workstations.
If you really must, you can, at the risk of bugs and problems that course personnel can probably not help you solve, attempt to use Windows or the even more unstable version of Gephi for Macintosh.

Learning goals

The main goal of this lab session is to become familiar with Gephi (experimental beta-software to visualize networks for research purposes) and its input format. At the end of this session you should be able to import and visualize raw network data with labeled nodes and labeled and/or weighted edges (directed or undirected), and you should understand how to map edge and node size and color to structural network properties such as the node degree. You should be able to export computed node data and be able to export a vector graphic PDF of your network. There is no deliverable for this lab session.

Instructions

1. Extract and run Gephi locally on a student workstation (running it from your homedirectory may be slow), for example on the local disk in /scratch/gephi. Installation instructions, if needed, can be found on the Gephi website.
2. Run Gephi by starting the Linux executable bin/gephi. If this does not immediately work, you may have to adjust one line in etc/gephi.conf so that Gephi can find the right Java version (and remove the # in front of the line):
   jdkhome="/usr/lib/jvm/java-8-openjdk-amd64/"
3. The Gephi website has various official tutorials that you can walk through; in particular Quick start, Visualization and Layouts.
   Note: The Quick Start Guide is for a slightly older version of Gephi. On p. 9-10, contrary to what is indicated in the getting started guide, the ranking module is part of the appearance module (since Gephi 0.9), and does not appear as a separate tab. The “Ranking result” table no longer exists, and you can skip that part of the tutorial. In the tutorial on Visualization, on p. 9 the 3-D visualisation option was bugged and has been removed in Gephi 0.9. You can skip that part of the tutorial.
   You may instead benefit from using a more modern tutorial (courtesy of Derek Geene at UCD).
4. Try out some of the Gephi datasets and see if you can understand how they differ in terms of density, degree distribution, giant components, clustering and average distance.
5. Learn how to import raw data from this short data import tutorial. Visualize the network represented by the edge list /vol/share/groups/liacs/scratch/SNACS/small-gephiready.tsv.
   Compute the degree for each node and export it to a file (which you can for example reuse in other plotting tools).
6. Make a new, very simple, edge list input file of a network with say 10 nodes and 20 edges, and import it into Gephi. Now add to the input file (in some order) link (un)direction, link weights, node labels and link labels. For example, have nodes represent students, and link represent friendships, and choose node and edge labels accordingly. Make sure that you are able understand the difference in file input and visualization output.
7. Play around with some different visualization algorithms such as ForceAtlas 2 (for example, with scaling set to a higher value and with stronger gravity checked) and Fruchterman Reingold. Export your colored visualization to PDF so that you have it as a vector graphic.
8. Get started with Exercise 2 (the practical part) of Assignment 1.