Portfolio Analytics
This web page is for internal DOE use only
Webinar Series on Portfolio Analytics – Tools for R&D Decision Making
Federal R&D agencies need effective tools for analysis of past investments and for placement of future investments. Such tools are often covered under the umbrella of “portfolio analytics.” This webinar series will feature needs, strategies, and tools used by several federal agencies to assess the impacts and results of past R&D investments; to identify trends and potential areas of collaboration; and to support strategic decision making for long-term funding requests. A perspective from academia will also be provided. Presenters will demonstrate the types of data needed for effective tools and examples of the questions being answered by these tools.
Table of Contents
Webinar 1
Portfolio Analytics Webinar 1
Using data science and artificial intelligence in data-driven management of research investments
August 11, 2020, 2:00 pm – 3:00 pm
George M. Santangelo, Ph.D., Director, Office of Portfolio Analysis, National Institutes of Health
Kristine A. Willis, Ph.D., Program Director, National Institutes of Health
Abstract
The Office of Portfolio Analysis (OPA) at NIH was established in 2011 to develop, validate, and disseminate new methods, computational tools, and best practices for portfolio analysis; generate high quality, fully interoperable databases; use those resources to answer important questions about the NIH portfolio; and train NIH staff to do the same. NIH investments, which span a complex landscape of topics as diverse as epidemiology, social psychology, tissue engineering, and physical chemistry, include more than 38,000 funded research project grants each year (totaling over $21.5B in FY2019). Optimizing management of this large portfolio requires a deep understanding of its inherent complexity. As part of the effort to achieve this optimization, OPA has developed a next-generation suite of web tools that delivers RCR (Relative Citation Ratio) values, APT (Approximate Potential to Translate) scores, and all raw input data to the desktops of decision‑makers and other stakeholders, including the public. Current OPA tools, which include iCite, iSearch, and most recently the COVID-19 Portfolio, also employ AI/ML, NLP, and other analytical schema that provide users with sophisticated tools for analyzing documents, metadata, and associated datasets. OPA also seeks to identify opportunities to encourage innovative and/or transformative science. In this presentation Dr. Santangelo will illustrate how three central components of the OPA mission are focused on promoting good stewardship of America’s investments in biomedical research:
- Using analytics to characterize portfolios of any size (tens to thousands of investments);
- Identifying trends, especially those that reveal paths to improved decision-making;
- Making successful predictions, including which topics will produce scientific breakthroughs
Dr. Willis will then describe how program staff benefit from OPA tools and other resources, and how data-driven decision-making can improve the efficiency and effectiveness of program management.
Bio
George Santangelo has been the Director of the Office of Portfolio Analysis (OPA) at NIH since 2011. He received his bachelor's degree in molecular biology from the University of Pennsylvania and his doctorate in genetics from Yale University. Before coming to NIH, George was a systems biologist and data scientist funded by both NIH and NSF to lead research teams investigating how genes are regulated. In OPA, George now leads a team of analysts, data scientists, and software developers whose mission is to learn how to accelerate the pace of scientific discovery and disseminate that knowledge across funding agencies and the research community.
Bio
Kris Willis has been a program director at the National Institutes of Health since 2014. She currently manages a $41M portfolio of grants on metabolism for the National Cancer Institute. Prior to joining NIH, Kris held a faculty position in the Department of Biology at Georgetown University. Her interest in applying analytics to portfolio management is deeply informed by her background in genomics and systems biology. She has collaborated with OPA on numerous projects, and has taught classes focused on the use of portfolio analysis tools by program directors, including a component of the NIH Core Curriculum, which is the mandatory series of introductory courses taken by all new agency employees.
Webinar 2
Portfolio Analytics Webinar 2
Modeling and Visualizing Science and Technology Developments
August 19, 2020, 11:00 am – 12:00 pm
Katy Börner, Victor H. Yngve Distinguished Professor of Engineering and Information Science Luddy School of Informatics, Computing, and Engineering, Indiana University
Abstract
In the information age, the ability to read and make data visualizations is as important as the ability to read and write. This talk introduces a theoretical data visualization framework (DVL) meant to empower anyone to systematically render data into insights using temporal, geospatial, topical, and network analyses and visualizations. Exemplarily, the DVL is applied to
- Map science and technology, see interactive data visualizations from the Places & Spaces: Mapping Science exhibit (http://scimaps.org) and recent PNAS special issue on Modeling and Visualizing Science and Technology Developments (https://www.pnas.org/modeling).
- Teach Visual Analytics (https://visanalytics.cns.iu.edu) to students around the globe.
References
- Börner, Katy, Andreas Bueckle, and Michael Ginda. 2019. Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments. PNAS, 116 (6) 1857-1864.
- Börner, Katy. 2015. Atlas of Knowledge: Anyone Can Map. Cambridge, MA: The MIT Press.
- Börner, Katy. 2010. Atlas of Science: Visualizing What We Know. Cambridge, MA: The MIT Press.
Bio
KATY BÖRNER is the Victor H. Yngve Distinguished Professor of Engineering and Information Science in the Departments of Intelligent Systems Engineering and Information Science, School of Informatics, Computing, and Engineering; core faculty of the Cognitive Science Program; and founding director of the Cyberinfrastructure for Network Science Center (http://cns.iu.edu)—all at Indiana University in Bloomington, Indiana.
She is also a visiting professor at the Royal Netherlands Academy of Arts and Sciences (KNAW) in the Netherlands; Visiting Professor and Mercator Fellow in User-Centered Social Media, Department of Computer Science and Applied Cognitive Science, University of Duisburg-Essen, Duisburg, Germany; and Humboldt Fellow, Technology/Media Centre, Dresden University of Technology, Germany. Börner became a Fellow of the American Association for the Advancement of Science (AAAS) in 2012, a Humboldt Research Fellow in 2017, and an Association for Computing Machinery (ACM) Fellow in 2018. Since 2005, she serves as a curator of the international Places & Spaces: Mapping Science exhibit (http://scimaps.org).
Börner’s research focuses on the development of data analysis and visualization techniques for information access, understanding, and management. She is particularly interested in the formalization, measurement, and systematic improvement of people’s data visualization literacy; the study of the structure and evolution of scientific disciplines; the analysis and visualization of online activity; and the development of cyberinfrastructures for large-scale scientific collaboration and computation.
She holds an MS in electrical engineering from the University of Technology in Leipzig, and a PhD in computer science from the University of Kaiserslautern.
Webinar 3
Portfolio Analytics Webinar 3
Advanced Research Projects Agency – Energy (ARPA-E) Impact Tracking and Analytics Platform Overview
September 10, 2020, 1:00 – 2:00 pm
Shane Kosinski, Deputy Director for Operations, Advanced Research Projects Agency-Energy (ARPA-E) and Patrick Finch, Lead Associate, Booz Allen Hamilton
Abstract
In 2016, a National Academies of Science (NAS) assessment recommended that the Advanced Research Projects Agency - Energy (ARPA-E) expand on its impact tracking framework to better understand how effectively the Agency was meeting its mission requirements, particularly in the area of driving innovation in energy technology space. In this talk, Shane Kosinski, ARPA-E's Deputy Director of Operations and Patrick Finch of Booz Allen Hamilton will be presenting on the Agency's response to this feedback: the creation of ARPA-E's Indicators Measuring Progress & Change Tracking System (IMPACTS). The IMPACTS platform helps ARPA-E create and use data in a meaningful way through ongoing data tracking, cleaning, and analysis. The platform has provided ARPA-E with a wide range of abilities to measure and assess impacts of this funding across its critical mission areas, including identification of potential impacts of new programs before award, monitoring of project management factors during award, and validation of research impacts after award. In addition, the team at ARPA-E has provided data analytics and data science subject matter expertise to identify relationships between fields and technologies and drive predictive analysis.
Bio
SHANE KOSINSKI is the Deputy Director for Operations for the Advanced Research Projects Agency-Energy (ARPA-E), responsible for oversight and operations of the mission of ARPA-E as well as all ARPA-E programs. In this role, Kosinski develops flexible management processes to create visibility for the clear oversight of more than 300 active research projects. As Head of Contracting Authority (HCA) for ARPA-E, Kosinski established and staffed a new independent procurement office completely internal to ARPA-E. In 2009, Kosinski was selected to lead the efforts of establishing the ARPA-E office as a new agency within the Department of Energy (DOE). As the first employee of ARPA-E and the Acting Deputy Director, Kosinski led ARPA-E’s first Funding Opportunity Announcement (FOA), which provided $150M to 37 transformational energy projects that could one day change the way the U.S. produces and uses energy. Prior to his work at ARPA-E, Kosinski worked in the DOE’s Office of the Chief Financial Officer. During this time, he led several agency-wide efforts for the 2009 Presidential Transition and the American Recovery and Reinvestment Act of 2009. Kosinski’s work focused on complex policy and financial issues impacting the Department. Kosinski has a B.S. in Economics and Biological Science from Binghamton University. He also earned a M.A. in Economics from the University at Albany.
PATRICK FINCH is a Lead Associate at Booz Allen Hamilton with over fourteen years of experience in the energy, financial, and data analytics sectors. His work has focused primarily on researching and analyzing the benefits and costs of clean energy and energy efficiency projects, and working with R&D teams to help their technologies attain commercial viability. He is currently engaged in supporting the U.S Department of Energy’s Advanced Research Projects Agency – Energy (ARPA-E) as the manager of their internal impacts analytics team, as well as serving as a Technology-to-Market Advisor for a broad portfolio of research projects. He holds a Master’s degree in International Trade and Investment Policy from the George Washington University and is an ICAgile Certified Professional.
Webinar 4
Portfolio Analytics Webinar 4
Overview of Portfolio Analytics at the Office of Naval Research
October 1, 2020, 2:00 – 3:00 pm
Dr. Julie Christodoulou, Director, Naval Materials Science and Technology (S&T) of the Office of Naval Research (ONR)
Dr. Ryan Zelnio, Chief Analytics Officer, ONR
Abstract
The Office of Naval Research established the Data & Analytics division in January 2018 to support data-driven decision making throughout the S&T portfolio. This talk will discuss major efforts undertaken to support this mission and highlight some of the successes over the past year. This talk will be followed by comments from Materials Division Director on how she has been utilizing these services.
Bio
DR. JULIE CHRISTODOULOU directs the Navy’s materials science and technology programs for efforts that span in technical maturity from basic research to demonstrations and manufacturing technologies, engaging a broad expanse of science and engineering disciplines. She coordinates research activities with colleagues domestically and internationally, serving as a Principal to the DoD Community of Interest for Materials and Manufacturing Processes, The Technical Cooperation Program Materials Group (an international organization of defense laboratories), the NSTC Subcommittee for the Materials Genome Initiative, and others. Trained as a physical metallurgist, Dr. Christodoulou has studied at the University of Texas, the Johns Hopkins University and Imperial College, London. She has worked in corporate research and at NRL, ONR-Global and NSWC-Carderock Division before joining ONR in 2002 as a program officer and taking on the role of division director in 2007
DR. RYAN ZELNIO is currently the Chief Analytics Officer for the Office of Naval Research where he develops new analytics to support strategic and tactical decision making on the Navy's Science & Technology portfolio to provide future capabilities to the Navy and Marine Corps. Prior government experience includes the Associate Director for Tech Watch and Horizon Scanning at the Office of the Undersecretary of Defense (Research & Development), an international programs office for ONR Global, and as a computational social scientist at NSWC-Dahlgren. Prior to government, he worked in a number of private sectors including space, finance and automotive as a computer scientist and as a research fellow at the National Academies. Dr. Zelnio received his Ph.D. in Public Policy from George Mason University, M.S. in Space Studies from University of North Dakota and a BS in Computer Science and Mathematics from Marycrest University.
Additional Resources
Demo of NSF’s Research Analytics Tool
Presentation by Paul Morris and Rebecca Keiser on NSF’s Research Analytics Tool and demonstration of potential applications as they relate to international cooperation and research security.
For inquiries, please contact Yvette Wallus, wallusy@osti.gov