Intel Advisor is an optimization tool that helps the developers identify hot spots, performance issues and also provide recommendations for performance improvement. It has been installed at most of the Princeton research computing systems. Intel Advisor was part of the licensed Parallel Studio XE (PSXE) releases before. It is now included in the Intel OneAPI base toolkit, which is free to download. In this article, we will walk you through the process of collecting performance data remotely at Princeton Research Computing clusters using the Intel Advisor command line interface (CLI) and displaying the results on a local macOS system using the Intel Advisor graphical user interface (GUI).
Preparing Applications for Performance Analysis
For C/C++ and Fortran code (on Linux OS), it is recommended to setup the following compiler flags before running the performance analysis:
Request full debug information (compiler and linker): -g
Request moderate optimization: -O2 or higher
Disable inter procedural optimization that may inhibit the profiler to collect performance data: -no-ipo
Using Intel Advisor on Princeton Research Computing Clusters
Before, we usually recommended that you used the CLI to collect data via batch jobs at compute nodes and then viewed results using the GUI on a login node. Now as the Intel Advisor GUI is available free on macOS, we recommend that you copy the collected data from the remote system to your local macOS to view. Note Intel Advisor does not support data collection on macOS and you can only use macOS for displaying the data collected on a Windows or Linux OS.
Collecting Data at Remote System
Once in a remote system (e.g., Tigercpu, Adroit etc), you start by loading the module, e.g., module load intel-advisor. Then you can collect the data using the Intel Advisor CLI. The CLI is launched with advisor command. You can use advisor –helpto search for the command for a specific action. For example, after issuing advisor –help command, you will see
Intel(R) Advisor Command Line Tool
Copyright (C) 2009-2020 Intel Corporation. All rights reserved.
Usage: advisor <--action> [--action-option] [--global-option] [[--] <target>
<action> is one of the following:
collect Run the specified analysis and collect data. Tip: Specify the search-dir when collecting data.
command Issue a command to a running collection.
create-project Create an empty project, if it does not already exist.
help Explain command line actions with corresponding options.
import-dir Import and finalize data collected on an MPI cluster.
mark-up-loops After running a Survey analysis and identifying loops of interest, select loops (by file and line number or criteria) for deeper analysis.
report Generate a report from data collected during a previous analysis.
snapshot Create a result snapshot.
version Display product version information.
workflow Explain typical Intel Advisor user scenarios, with corresponding command lines.
For help on a specific action, type: advisor --help <action>
Perform a Survey analysis.
advisor --collect=survey --project-dir=./advi --search-dir src:r=./src -- ./bin/myApplication
Generate a Survey report.
advisor --report=survey --project-dir=./advi --search-dir src:r=./src
Display help for the collect action.
advisor --help collect
advisor –help collect shows you the command to perform a specific analysis. For example, to perform a survey analysis to determine hotspots, we use
It is also helpful to use the GUI to find out the command. For example, you can:
Log in a remote head node with ssh -Y firstname.lastname@example.org
Load the module with module load intel-advisor
Launch the Intel Advisor GUI with advisor-gui
Create a project
Set up the project properties
Choose the appropriate analysis type
Click the get command line button on the workflow tab under the desired analysis
Copy the command line to clipboard to paste to the script for remote runs
To view the results, you can copy the whole project directory to your local macOS. It is also recommended to first pack the analysis results in a snapshot and then copy the packed *.advixeexpz file. For example:
You can download the Intel Advisor for macOS from the oneAPI base toolkit. After launching the Intel Advisor GUI, you then go to File > Open > Project/Result and navigate the copied project directory/snapshot.
This article covers the following NEW in Intel Advisor version 2021.1:
Intel Advisor is included as part of the Intel OneAPI base toolkit
The executables are renamed. advixe-cl is renamed to advisor. advixe-gui is renamed to advisor-gui
The roofline analysis is provided as a single command. In the earlier version, roofline analysis is done by first running a survey analysis followed by a tripcounts analysis. Now we can run the roofline in a single step using —collect=roofline option.
As a researcher, inquiries about previously published research probably evoke two feelings: panic-filled regret or calm authority. Often the difference is time; it’s easier to talk about the project you worked on last week than last decade. Talking about old protocols or software is a lot like someone critically examining the finger painting you did as a child. You know it’s not perfect and you would do several things differently in hindsight, but it is the method in the public record. The rate of change seems faster with software development, where new technologies redefine best practices and standards at dizzying rates.
Perhaps the most challenging problem is when researchers outside of your institution fail to reproduce results. How can you troubleshoot the software on every system or determine what missing piece is required to get things working? What was that magic bash command you wrote 5 years ago?
Blocklint is a simple command line utility for finding non-inclusive wording with an emphasis on source code. If you’ve used a modern IDE, you know the importance of immediate feedback for compilation errors or even stylistic slip-ups. Knowing all variables should be declared or that lines must be less than 80 characters long is good, but adhering to those rules takes a back seat when in the flow of writing code. A linter brings these issues back into your consciousness by highlighting the problematic lines of code. Over time, the enforced style becomes more intuitive but the linter is always there to nudge you if you slip.
I started working as a research software engineer for the Princeton Neuroscience Institute (PNI) in May 2017. At the end of my first week I received an email from Professor Mala Murthy and post-doc David Deutsch of the MurthyLab, asking if I would be interested in working on a project involving a “virtual reality environment for neural recording experiments”. The kid in me got very excited at the prospect of making video games. At the time I did not know the project was to develop a virtual reality simulation for flies!
Virtual reality experiments have a long history in neuroscience . They allow researchers to restrict the movement of animal subjects so that they can use advanced microscopy to image their brains in “naturalistic” environments. In the Murthy lab’s VR setup, the fly is fixed to the objective of a two photon microscope and suspended above a small sphere floating on a column of air. This small sphere is used as a sort of omni-directional treadmill. While the fly cannot actually move, it can move its legs, which in turn move the freely rotating sphere. The movement of the sphere is tracked with computer vision algorithms and thus a fictive path for the fly in a virtual world can be reconstructed. This setup allows a “moving” fly’s brain to be imaged with techniques that require it to be stationary. The two photon imaging system then provides a very flexible and powerful tool for studying changes in the fly’s brain activity over time of the experiment. Different spatial and temporal resolutions are available depending on the needs of the experimenter.
As I started using Snakemake, I had hundreds of jobs that I wanted to get performance information about. seff gives the efficiency information I wanted, but for only a single job at a time. sacct handles multiple jobs, but couldn’t give the efficiency. With the current python implementation of reportseff, all job information is obtained from a single sacct call and with click the output is colored to quickly see how things are running.
This Python library is part of a larger picture in the Scikit-HEP ecosystem of tools for Particle Physics and is funded by DIANA/HEP and IRIS-HEP. It is the core library for making and manipulating histograms. Other packages are under development to provide a complete set of tools to work with and visualize histograms. The Aghast package is designed to convert between popular histogram formats, and the Hist package will be designed to make common analysis tasks simple, like plotting via tools such as the mplhep package. Hist and Aghast will be initially driven by HEP (High Energy Physics and
Particle Physics) needs, but outside issues and contributions are welcome and encouraged.
As with any substantial application package, the ASPIRE project needed a convenient way to specify configuration settings pertaining to different parts of the computational pipeline.
What follows below are some outlines from our attempts to tackle this configuration issue. Where a supplementary (and hopefully useful) nugget is provided, or a caveat discussed, I shall append a linked numeral, like so: (n)
A brief background of ASPIRE
ASPIRE is a Python (3.6) package under development, which ingests Micrographs, the output of Cryo-Electron Microscopy (images that closely resemble television static), and comes up with a 3D reconstruction of the molecule. Read the excellent writeup on the ASPIRE page for a more comprehensive review of the package.
APPLE-Picker is a submodule of ASPIRE Python package in development for reconstructing a 3D CryoEM map of biomolecule from corresponding 2D particle images, developed by the researchers in Professor Amit Singer’s group. It is an automatic tool to select millions of particles from thousands of micrographs, a critical step in the pipeline of CryoEM image reconstruction. It used to be performed manually but can be very tedious and difficult especially for small particles with low contrast (low signal-noise ratio). The CPU version takes ~80 seconds on average to finish processing one micrograph. To achieve the goal of finishing thousands of micrographs in a few minutes, we need an alternative method, such as GPU accelerating.
2019 Princeton GPU Hackathon
Princeton university held its first GPU hackathon on campus this summer from June 24 to 28, organized and hosted by the Princeton Institute for Computational Science and Engineering (PICSciE), and co-sponsored by NVIDIA and the Oak Ridge Leadership Computing Facility (OLCF). The main goal of this Hackathon was to port research codes to GPUs or optimize them with the help of experts from industry, academia and national labs, as emphasized by Ian Cosden, one of lead organizers and manager of Princeton’s Research Software Engineering Group. This blog reports our attempts and the story behind accelerating APPLE-Picker using GPU and parallel computing in Python.
Background on ASPIRE (Algorithms for Single Particle Reconstruction)
Significant progress on computational algorithms and software is one of the major reasons leading the revolution of resolution in three dimensional structure determination of biomolecules using CryoEM, a technique projecting rapidly frozen and randomly orientated 3D particles into 2D noisy images on micrographs and reconstructing 3D density maps in atomistic resolution through computer software. Due to many crucial roles of 3D biomolecules such as protein enzymes for further study in structural and chemical biology, biophysics, biomedical and other related fields, the 2017 Nobel prize in chemistry was awarded to three scholars for significantly advancing the CryoEM technique as explained in this Youtube video.
During the past 10 years, Professor Amit Singer’s group has proposed many new ideas in various numerical algorithms and developed the ASPIRE Matlab package to tackle many problems involved in reconstructing a 3D CryoEM map of biomolecule from corresponding 2D particle images, including CTF estimation, denoising, particle picking, 2D and 3D classification, and ab initio 3D reconstruction.
Welcome to the first Princeton RSE group blog entry! Before we get into the good stuff, here’s a bit of background
A relatively new addition, the Princeton RSE group formed in late 2016 within the Princeton Research Computing Department. We develop software long term with traditional research groups to enable and advance research. We strive to develop high quality software both in terms of performance, and sustainability/maintainability. We work alongside research groups from multiple disciplines. You can read more about our group on our webpage and the individual members of our group here.
What you can expect out of this blog
We’ll be sharing software projects we’ve worked on including major releases, scripts and solutions we’ve generated, and little interesting discoveries that have come up along the way. We’re looking to share some of the best practices we’ve settled on, point out some lessons learned, and try to foster discussions within the broader RSE and research software community.