A Review of Wildbook: Software to Support the Environment

This is a guest post by Carmen Camp. See Carmen’s bio at the bottom of this post

Wildbook is an open source software platform devoted to tracking animals in the wild and decreasing the chances of extinction in different species. The software itself, WildMe, is publicly available on GitHub to encourage people to join the movement in protecting wildlife. Much of the success of the program relies on citizen science and interaction, and the system itself utilizes artificial intelligence to identify animals and other environmental features from photographs. A unique advantage of using photos for identification purposes is that when an individual animal is encountered in the wild, today’s digital cameras are able to capture a large number of high quality photographs before the encounter is over. These photos may then be used to further improve animal identification by using machine learning software that is available through Wildbook.

Wildbook processes photos from research teams, social media, and citizens alike. Wildbook uses deep convolutional neural networks to analyze a photograph, and identify animals, plants, and other objects contained within the scene. The system’s pattern matching algorithms are also capable of identifying unique individuals that are known to research teams. This gives scientists important opportunities to track individuals, as well as populations, and social interactions of the animals.

Early Work with Whale Sharks

The purpose of Wildbook is to be an open-source platform that enables different research teams to perform photo identification with limited manual labor. Since its beginnings, Wildbook has been used by many projects, one of the first of which was a 2005 project to study whale sharks. The photo identification software was based on software to identify constellations (Arzoumanian et al., 2005). Once adapted, the software enabled researchers to identify whale sharks from their size and shape, as well as their spot patterns. Spot coordinates are represented in XML and either match known individuals, or are used to identify previously unrecorded individuals.

A later study in 2008 also used a whale shark database along with the lab’s mark-recapture methods to better understand the survival rates of whale sharks in Western Australia (Holmberg et al., 2008). The 2005 and 2008 studies, and others, have led to a “Wildbook for Whale Sharks” organization that is dedicated primarily to keeping track of whale sharks around the world. The organization’s website has a featured link that allows users to “Report your sightings,” which requests input about the user, the sighting, its date and location, and whatever footage or photos the user captured of the whale sharks.

Improvements in the Wildbook for Whale Sharks software has spread coverage of whale sharks from Western Australian populations, to the Philippines, to the Western Atlantic Ocean. While some studies focus on geographic areas and the distribution of the animals (Araujo et al., 2016), others track specific animals (Norman and Morgan, 2016). Still other projects track social groups and seasonal migrations of different populations, like the 2013 study on sharks in the Gulf of Mexico and the Caribbean Sea (Hueter et al., 2013).

Extending Wildbook to other Species

Not only can an organization for an individual species like whale sharks use Widbook, but citizens and scientists alike can participate in the research by providing photographic evidence of many types of animals. Manta Matcher is another example of programs that have been able to form because of the capabilities offered by Wildbook. Its website has a nearly identical setup to Wildbook for Whale Sharks, and it provides easy access to a page through which users can submit data and photographs they collected. Flukebook is yet another option for photo identification that is, as its name might suggest, an identifying software for the flukes of whales. Each of these programs that are directly related to Wildbook offer resources that in turn are used by many scientific organizations. For example, the Dominica Sperm Whale Project uses Flukebook to keep track of individuals in the island’s surrounding waters. Shane Gero, a scientist from the project, is quoted on WildMe’s website, saying, “PhotoID as a tool for conservation and research finds power in numbers and international, inter-institutional collaboration,” indicating that a globally collaborative platform like Wildbook is exactly what the scientific world needs to get answers and solve problems.

Some programs use Wildbook to find individual known animals, rather than categorizing creatures as one species or another. In 2013, an algorithm was introduced called HotSpotter, which used pattern matching methods to identify unique sections on photographed animals. Thus, HotSpotter became a versatile option for identifying multiple types of animals, as well as individuals within the same species. HotSpotter focuses on identifying key points on animals in the frame of the photograph, and it then uses a nearest-neighbor search to compare the new photo with pre-existing records in the database of individual animals. To train and test the system, it was run on photos from scientists, assistants, ecotourists, and ordinary citizens. The software has been successfully used on many creatures, including two types of zebras, giraffes, leopards, and lionfish (Crall et al., 2013).

Citizen Scientists and Wildbook

Every example of Wildbook usage for external research highlights the importance of citizen interactions that make the research possible. Machine learning schemes and artificial intelligence typically require large amounts of data with which to be trained, and in order to test the algorithms, completely fresh, non-repeated data files must be used. This means that in order for an algorithm to be effective and accurate, it must have a huge source of data files, which in this case are photographs. Thus, citizens’ and the public’s interactions with projects like Wildbook are essential to success. This necessity for external input is evident in many related pieces of research, such as a 2017 case study on Twitter. This study discusses the crucial role that everyday members of the public play in data collection. In this study, the researchers wanted to train a machine learning algorithm to identify emotions in tweets on Twitter. The people on the research team, however, could not manually provide enough examples and training data to have a fully functioning algorithm. It simply would take far too long to be worthwhile. Using citizen scientists, however, the team was able to gather enough data to get their algorithm running accurately in a reasonable amount of time (Sastry et al., 2017).

Not only is advancing technology providing more opportunities for the public to get involved with research, but it is also offering new and more accessible ways of participating in different projects. Take, for example, the Humane Society, which asks citizens to submit information about roadkill they come across online. The National Audubon Society also has a program for volunteers to count and monitor birds for an annual census. Programs such as these, as well as Wildbook spinoffs, allow citizen scientists to submit vital information through the internet or applications. Evolving technology has placed the ability to submit such data directly in our hands through mobile phones and other devices, and the internet offers places to submit data, as well as aids to citizens in finding causes and projects with which they can help.

WildMe’s website divides citizen scientists into four distinct groups that are categorized by the role the citizens play in research. The first group are denoted as “scientists.” These citizens are incredibly engaged in the research, and they focus their efforts on analyzing data and determining its meaning. Second are the “evangelists,” who are devoted to outreach and explaining the research project to the public. They play an important role in motivating more people to join them in the effort, as well as building communities that support the research. The next role is “the technologist,” which further emphasizes the significance of technological advancements. These people make sure that the IT side of the project allows it to be as efficient and interactive as possible. Finally, “the volunteer” educates members of the public so that they are capable of collecting data, monitoring inputs, or analyzing information, as it may relate to the research.

Wildbook in the Global Community

Those in charge of Wildbook also understand the power of globalization and how the world is connected through the internet. With an active Twitter account, the organization is able to advertise itself through new pieces of research that use the software. The Wildbook Facebook page is also active. It provides similar updates to Twitter, as well as chances for visitors to donate to the cause, attend related events, and participate in virtual reality activities to learn more about the animals. For example, The “Great Grevy’s Rally” is currently publicized on the Facebook page, and it invites people to go to Kenya in January to help complete a census on the Grevy’s zebras in the area. It welcomes any and all aspiring citizen scientists to join the charge in driving around a designated area to photograph and document and zebras seen in the area. The data collected will then be put into the Grevy’s zebra Wildbook database. The Facebook link on the event page redirects one to a page describing the Grevy’s zebra mission, and clearly offering tips on how to become a citizen scientist and help the cause. By marketing these events in places that supporters are likely to see them, Wildbook gains both support and renown via the internet.

Wildbook provides an incredible opportunity to globalize scientific and ecological missions like was never before possible. Individuals of any profession from around the world can participate in the global mission to save and preserve the planet on which we live. The software provides an interface between science and the public that is easily accessible, especially for individual species that have their own associated organizations and easily accessible databases and websites. From whale sharks, to giraffes, to leopards, to lionfish, Wildbook has introduced endless options for collaboration. More than simply using user-generated photographs, Wildbook offers people the option to get involved and have a part in science, which is a crucial piece of gathering a force together that can have a positive impact on our changing world and dangerously shifting animal populations.

Carmen Camp will graduate in spring 2018 with a degree in Computer Science and Corporate Strategy. She is passionate about marine science and hopes that her future will include plenty  of opportunities to help protect the ocean. She may be contacted at carmen.camp@vanderbilt.edu.


Araujo, G., Snow, S., So, C. L., Labaja, J., Murray, R., Colucci, A., and Ponzo, A. (2016) Population structure, residency patterns and movements of whale sharks in Southern Leyte, Philippines: results from dedicated photo-ID and citizen science. Aquatic Conserv: Mar. Freshw. Ecosyst., doi: 10.1002/aqc.2636.

Arzoumanian Z, Holmberg J & Norman B (2005) An astronomical pattern-matching algorithm for computer-aided identification of whale sharks Rhincodon typus . Journal of Applied Ecology 42, 999-1011.

Hueter RE, Tyminski JP, de la Parra R (2013) Horizontal Movements, Migration Patterns, and Population Structure of Whale Sharks in the Gulf of Mexico and Northwestern Caribbean Sea. PLoS ONE 8(8): e71883. doi:10.1371/journal.pone.0071883

Holmberg J, Norman B & Arzoumanian Z (2008) Robust, comparable population metrics through collaborative photo-monitoring of whale sharks Rhincodon typus. Ecological Applications 18(1): 222-223.

J. P. Crall, C. V. Stewart, T. Y. Berger-Wolf, D. I. Rubenstein and S. R. Sundaresan, “HotSpotter — Patterned species instance recognition,” 2013 IEEE Workshop on Applications of Computer Vision (WACV 2013), pp. 230-237.

Norman B. and Morgan D. (2016) The return of “Stumpy” the whale shark: two decades and counting. Front Ecol Environ 2016; 14(8):449–450, doi:10.1002/fee.1418

Sastry, N., et al.: “Bridging big data and qualitative methods in the social sciences: A case study of Twitter responses to high profile deaths by suicide,” Online Social Networks and Media (2017)

The Importance of Citizen Scientists

Wildbook: http://www.wildbook.org/doku.php?id=start

RegionRadio and Related Works

This is a post by Emily Markert. See her bio at the bottom of this post.

Many of us have experienced car rides, both short and long, filled with small-talk, music played from a radio, or even podcasts. Although these media seem to do well enough at keeping us entertained, they often contribute little to any higher purpose. The vast amount of time people spend in transit has great potential to provide a meaningful experience, yet typically goes unutilized. Thus, we introduce RegionRadio, a spatially-aware storytelling tool aimed at supporting sustainability efforts through place-based education. The theory behind place-based education argues that immersing people in the history of a place can increase the connection they feel to it, and therefore increase the likelihood that they would act to protect it. In this way, RegionRadio exhibits place-based education by presenting stories to a user as they travel along a route; these stories are mandated to be relevant to the user’s current geospatial location, to contain topics related to environmental concepts, and eventually, to be customized to specific users’ preferences and histories. These stories are analyzed and filtered based on their topic, perceived interestingness, and continuity of the ‘playlist’ presented to the user throughout their journey. Use of RegionRadio is intended to be as simple as turning on the radio, but aims to develop ordinary citizens as environmental advocates by heightening appreciation of their surroundings through the telling of stories.

Although this project is unique in its sourcing and presentation of environmentally-focused place-based education,  we take inspiration from other work in spatially-based education, to include SCRABS and DETOUR.

The SCRABS system was designed by a group of researchers from the Cultural Heritage Information Systems national project, and is intended to increase cultural appreciation by presenting users with personalized, context-based information as they explore historic sites. The system has not yet been fully developed, but presents an ambitious design that can be further reviewed in this publication. This context-dependent recommendation of information aimed at increasing cultural appreciation is highly relevant to our own task, but applied to a different domain.

We consider another tool that is fully-developed, and has potential to convey place-based education. Detour is a growing platform offering the creation and taking of a novel type of audio tour. Taken on the user’s own smartphone, Detours can be taken in both indoor and outdoor spaces, and progress as the user moves through space, triggering narrations and other media along the way. While most existing tours published with Detour focus on urban areas, there is great potential in creating Detours for more natural settings. Allowing users to download tours that might lead them on a hike, through a National Park, or on a scenic drive would make place-based education easily accessible for Detour’s thousands of users.

To keep up with RegionRadio, stay tuned to this blog.

Region Radio a collaboration between the CompSustNet  lab at Vanderbilt University and the Space, Learnng, and Mobility lab at Vanderbilt University.  Research and development of Region Radio is supported by NSF Award #1521672 “Collaborative Research: CompSustNet: Expanding the Horizons of Computational Sustainability” and NSF Award #1623690 “EXP: Bridging Learning in Urban Extended Spaces (BLUES) 2.0

Emily Markert is a Computer Science undergraduate at Vanderbilt University. The opinions expressed herein are Emily’s and not necessarily those of Cornell University. You can reach Emily at emily.markert@vanderbilt.edu.

Creating Citizen Science Projects

This is a post by Mateus Winelmann. See Mateus’ bio at the bottom.

We often hear about major scientific discoveries in the media, like how scientists at CERN found the Higgs Boson or that a lifesaving drug like Harvoni, which can cure hepatitis C, is going to market. These are amazing breakthroughs, but oftentimes the process of discovery feels like something far removed from us. After all, most of us don’t have the training or resources to work on these kinds of projects. When cutting edge scientific research often requires years of specialized education and billions of dollars in funding (finding the Higgs Boson is estimated to have cost over US$13 billion), it doesn’t seem like something ordinary people can be a part of. That doesn’t have to be the case though! Citizen science consists of research done largely by the general public, typically without any significant cost for participants, and it can be incredibly valuable. To illustrate that, let’s talk about eBird.

eBird was launched in 2002 by the Cornell Lab of Ornithology and the National Audubon Society, where the idea is to allow people to document their bird sightings. Bird watchers already tend to keep records of the birds they see or hear, and eBird allows them to make those observations available to educators and researchers across the world. Making that data available is already bearing fruit, with ornithologists at Cornell publishing a paper last year discussing the migratory strategies of birds, which is discussed in a New York Times article. The paper’s authors were able to document how different species of birds from different parts of the country migrate in different ways thanks to eBird. According to one of the paper’s articles, it would have cost researchers millions of dollars to collect this data through traditional tracking methods, and even then, the data would not have been as detailed.

You can learn more about eBird and how to contribute here. If you are interested in finding other citizen science projects, take a look at NatureNet and SciStarter. If you’re feeling particularly ambitious, you can consider starting your own citizen science project. There are several tools and frameworks out there that can help you create and share a citizen science project, some of which are described here, and a few others I would suggest looking at are CitSci, crowdcrafting, Zooniverse, and iNaturalist. Starting your own project can be an ambitious undertaking, so I’d recommend taking the time to explore what projects are already out there to get an idea about how you might create your own project. While this page is directed at educators, it highlights a few important things to consider if you decide to create your own project. There is no shortage of things to be studied, so it’s just a matter of finding, or starting, a project that interests you.

Mateus Winelmann is a senior undergraduate student at Vanderbilt University. The opinions expressed herein are Mateus’s and not necessarily those of Cornell University. You can reach Mateus at mateus.winkelmann@vanderbilt.edu.

Livestock Insurance in Africa

This is a post by Emily Markert. See her bio at the bottom of this post.

For pastoralists in East Africa, weather is key.  The threat of drought is recurrent on the African rangelands, and has the potential to kill vast numbers of livestock, throttling herders and their families into poverty.  This uncertainty has been a long-term concern in the region, and has led researchers Andrew Mude, Chris Barrett, and Michael Carter to develop a technology-based insurance program to protect these herders.  This Index-Based Livestock Insurance uses satellite data to monitor weather conditions in pastoral regions, and estimates livestock deaths.  Herders receive payouts based on these predictions.

A team led by Carla Gomes, Director of the Institute for Computational Sustainability, has also developed mobile applications that allow herders to report conditions, introducing an element of citizen science.  The program has been implemented in multiple countries, and this innovative combination of technology and finance has proven to be a success.  The researchers behind this insurance have received numerous awards for their efforts, and countless pastoralists have seen their livelihoods stabilized.  More information on this program can be found in this article from the Cornell Chronicle, in a press release on Dr. Andrew Mude’s receipt of the 2016 Norman Borlaug Award for Field Research and Application, or in this blog post.

Emily Markert is a Computer Science  undergraduate at Vanderbilt University. The opinions expressed herein are Emily’s and not necessarily those of Cornell University. You can reach Emily at emily.markert@vanderbilt.edu.

Awards for Computational Sustainability Papers at AAAI-17 and IAAI-17

Computational sustainability has been a special track at AAAI since 2011. The track invites “research papers on novel concepts, models, algorithms, and systems” at the nexus of AI, and environmental and societal sustainability. The 2017 special track cochairs were Bistra Dilkina of Georgia Institute of Technology and Sabine Storandt of Julius-Maximilians-Universität Würzburg.

There were two CompSust awards given at the 2017 AAAI conference. The AAAI CompSust Best Paper Award was given to
Xiaojian Wu, Akshat Kumar, Daniel Sheldon, and Shlomo Zilberstein for “Robust Optimization for Tree-Structured Stochastic Network Design“. The CompSust Best Student Paper was given to Jiaxuan You, Xiaocheng Li, Melvin Low, David Lobell and Stefano Ermon for their paper “Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data.

The AAAI CompSust Best Paper Committee included Alan Mackworth of the University of British Columbia, Zico Kolter of Carnegie Mellon UIniversity, and Amy McGovern of the University of Oklahoma.

The IAAAI conference, co-located with AAAI, also had computational sustainability representation, and CompSustNet researchers received an IAAI-17 Deployed Application Award for
Phase-Mapper: An AI Platform to Accelerate High Throughput Materials Discovery” by  Yexiang Xue, Junwen Bai, Ronan Le Bras, Brendan Rappazzo, Richard Bernstein, Johan Bjorck, Liane Longpre, Santosh K. Suram, Robert B. van Dover, John Gregoire, and Carla P. Gomes.

Finally,  CompSustNet Executive Council member co-authored  the AAAI-17 Outstanding Paper “Label-Free Supervision of Neural Networks with Physics and Domain Knowledge” by Russell Stewart and Stefano Ermon. The paper describes research on using constraints to reduce the need for labeled data when learning to recognize and track objects. While not a computational sustainability paper per se, it a line of research that has clear implications for computational sustainability.

AAAI-17 CompSust co-chair Bistra Dilkina presents Best Paper award to Xiaojian Wu and Akshat Kumar, for their paper with Daniel Sheldon and Shlomo Zilberstein.


A complete list of computational sustainability representation at AAAI and IAAI is broader than the CompSust special track, and relevant papers from AAAI, IAAI, and the AAAI-17 Workshop on AI+OR for the Social Good (appended at the end) are listed here as a convenience. This list includes CompSustNet members and collaborators too, highlighted in blue.

Sunday, February 5
EAAI-17 Blue Sky Ideas in AI Education from the New and Future AI Educator 
Sun 5-5:50, Golden Gate 1-2
  • AI Education through Real-World Problems by Mark Crowley 
Monday, February 6
AIW1: Crowdsourcing Techniques and Methodologies  
Mon 10-11, Continental 9
  • Poster 1475: Species Distribution Modeling of Citizen Science Data as a Classification Problem with Class-Conditional Noise by Rebecca A. Hutchinson, Liqiang He, Sarah C. Emerson
GTEP1: E-Commerce and Auctions 
Mon 10-11, Golden Gate 6
  • Poster 2250: Proper Proxy Scoring Rules by Jens Witkowski, Pavel Atanasov, Lyle H. Ungar, Andreas Krause
IAAI-17: Transportation: AI Applied to Safer and More Efficient Travel 
Mon 11:30-12:30, Golden Gate 5
  • Risk-Aware Planning: Methods and Case Study on Safe Driving Routes  by John Krumm, Eric Horvitz
  • Predicting Fuel Consumption and Flight Delays for Low-Cost Airlines  by Yuji Horiguchi, Yukino Baba, Hisashi Kashima, Masahito Suzuki, Hiroki Kayahara, Jun Maeno
  • Determining Relative Airport Threats from News and Social Media  by Rupinder P. Khandpur, Taoran Ji, Yue Ning, Liang Zhao, Chang-Tien Lu, Erik R. Smith, Christopher Adams, Naren Ramakrishnan
PS1: Planning
Mon 2-3:30, Plaza A
  • Poster 790: Three New Algorithms To Solve N-POMDPs by Yann Dujardin, Tom Dietterich and Iadine Chadès
VIS3: Object Recognition 
Mon 2-3:30, Golden Gate 7-8
  • Poster 2845: Extracting Urban Microclimates from Electricity Bills by Thuy Vu, D. S. Parker
MLA3: Machine Learning Applications   
Mon 2-3:30, Golden Gate 3
  • 1289: Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction Junbo Zhang, Yu Zheng, Dekang Qi
IAAI-17: Deployed AI Systems 
Mon 2-3:30, Golden Gate 5
  • Phase-Mapper: An AI Platform to Accelerate High Throughput Materials Discovery  by Yexiang Xue, Junwen Bai, Ronan Le Bras, Brendan Rappazzo, Richard Bernstein, Johan Bjorck, Liane Longpre, Santosh K. Suram, Robert B. van Dover, John Gregoire, Carla P. Gomes
AAAI-17 Invited Panel: AI for Social Good
AAAI  Mon 4-5, Continental 4-6
  • Panelists: Eric Horvitz, Peter Mockel, Lynne Parker, and Gideon Mann. Moderated by Milind Tambe.
PS2: Deterministic Planning 
Tue 10-11, Golden Gate 1-2
  • Matrix Factorisation for Scalable Energy Breakdown by Nipun Batra, Hongning Wang, Amarjeet Singh, Kamin Whitehouse
Senior Member Talks 1 (Summary)  
Tue 11:30-12:30, Continental 9
  • 3441: A Selected Summary of AI for Computational Sustainability by Douglas H. Fisher
ML12: Methods 
Tue 11:30-12:30, Plaza A
  • 788: On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systemsby Besmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
GTEP6: Game Theory
Tue 2-3:30, Golden Gate 6
  • Poster 23: Algorithms for Max-Min Share Fair Allocation of Indivisible Chores by Haris Aziz, Gerhard Rauchecker, Guido Schryen, Toby Walsh
MLA6: Deep Learning / Neural Networks 
Tue 2-3:30, Golden Gate 3
  • Poster 629: Combining Satellite Imagery and Open Data to Map Road Safety by Alameen Najjar, Shuníchi Kaneko, Yoshikazu Miyanaga
  • Poster 573: Regularization in Hierarchical Time Series Forecasting with Application to Electricity Smart Meter Data by Souhaib Ben Taieb, Jiafan Yu, Mateus Neves Barreto, Ram Rajagopal
ML15: Reinforcement Learning
Tue 2-3:30, Plaza A
  • Oral 2097: Maximizing the Probability of Arriving on Time: A Practical Q-Learning Method by Zhiguang Cao, Hongliang Guo, Jie Zhang, Frans Oliehoek and Ulrich Fastenrath
VIS6: Videos
Tue 2-3:30, Golden Gate 7-8
  • Poster 1854: Counting-Based Reliability Estimation for Power-Transmission Grids by Leonardo Duenas-Osorio, Kuldeep S. Meel, Roger Paredes, Moshe Y. Vardi
IAAI:17: Smart Environments: Using AI Systems to Improve Day-to- Day Life 
Tue 2-3:30, Golden Gate 5
  • Crowdsensing Air Quality with Camera-Enabled Mobile Devices by Zhengxiang Pan, Han Yu, Chunyan Miao, Cyril Leung
  • Real-Time Indoor Localization in Smart Homes Using Semi-Supervised Learning by Negar Ghourchian, Michel Allegue-Martinez, Doina Precup
  • ParkUs: A Novel Vehicle Parking Detection System by Pietro Carnelli, Joy Yeh, Mahesh Sooriyabandara, Aftab Khan
MLA8: Applications of Supervised Learning  
Tue 4-5, Golden Gate 3 
  • 1751: Predicting Demographics of High-Resolution Geographies with Geotagged Tweets  by Omar Montasser, Daniel Kifer
ML17: Classification and Clustering   
Tue 4-5, Plaza A
  • 353: POI2Vec: Geographical Latent Representation for Predicting Future Visitors by Shanshan Feng, Gao Cong, Bo An, Yeow Meng Chee
IAAI-17: Decision Support: AI for Better Decision Making
Wed 10-11, Golden Gate 5
  • Cracks Under Pressure? Burst Prediction in Water Networks Using Dynamic Metrics by Gollakota Kaushik, Abinaya Manimaran, Arunchandar Vasan, Venkatesh Sarangan, Anand Sivasubramaniam
  • Optimal Sequential Drilling for Hydrocarbon Field Development Planning by Ruben Rodriguez Torrado, Jesus Rios, Gerald Tesauro
SCS1: Constraint Satisfaction  
Wed 11:30-12:30, Golden Gate 7-8
  • Poster 1556: General Bounds on Satisfiability Thresholds for Random CSPs via Fourier Analysis  by Colin Wei, Stefano Ermon
RU2: Sequential Decision Making 
Wed 11:30-12:30, Golden Gate 4
  • Poster 2970: Hindsight Optimization for Hybrid State and Action MDPs by Aswin Raghavan, Scott Sanner, Roni Khardon, Prasad Tadepalli, Alan Fern
STCOMPS1: Dynamic and Spatiotemporal Systems
Wed 2-3:30, Golden Gate 4
  • Oral 2042: Fast-Tracking Stationary MOMDPs for Adaptive Management Problems by Martin Péron, Kai Helge Becker, Peter Bartlett, Iadine Chadès
  • Oral 823: Robust Optimization for Tree-Structured Stochastic Network Design by Xiaojian Wu, Akshat Kumar, Daniel Sheldon, Shlomo Zilberstein
  • Oral 3171: Dynamic Optimization of Landscape Connectivity Embedding Spatial-Capture-Recapture Information by Yexiang Xue, Xiaojian Wu, Dana Morin, Bistra Dilkina, Angela Fuller, J. Andrew Royle, Carla P. Gomes
  • Oral 2303: Spatial Projection of Multiple Climate Variables Using Hierarchical Multitask Learning by André R. Gonçalves, Arindam Banerjee, Fernando J. Von Zuben
  • Oral 2412: Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data by Jiaxuan You, Xiaocheng Li, Melvin Low, David Lobell, Stefano Ermon
MLA11: Machine Learning Applications  
Thu 11:30-12:45, Golden Gate 6
  • Oral 1922: Fine-Grained Car Detection for Visual Census Estimation by Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Li Fei-Fei
ML25: Recommender Systems
Thu 11:30-12:45, Continental 1-3
  • Oral 1502: Polynomial Optimization Methods for Matrix Factorization by Po-Wei Wang, Chun-Liang Li, J. Zico Kolter

Workshop on AI+OR for Social Good

14:00-14:35: Invited Talk: Daniel Sheldon, University of Massachusetts Amherst and Mount Holyoke College, “AI for Ecology and Conservation”

14:35-14:55: Sean Mcgregor, Rachel Houtman, Claire Montgomery, Ronald Metoyer and Thomas Dietterich, “Factoring Exogenous State for Model-Free Monte Carlo”

14:55-15:15: Yiqun Xie, Kwangsoo Yang, Shashi Shekhar, Brent Dalzell and David Mulla, “Spatially Constrained Geodesign Optimization (GOP) for Improving Agricultural Watershed Sustainability”

15:15-15:35: Rui Zhang, Jefferson Huang and Tarun Kumar, “Preventive Leak Detection for High Pressure Gas Transmission Networks”

Computational Sustainability Virtual Seminar Series launches!

Dear Colleagues,

Stefano Ermon
Stanford Assistant Professor Stefano Ermon

We are pleased to kick-off the Computational Sustainability Virtual Seminar Series with a talk by Professor Stefano Ermon of Stanford University, co-author on ”Combining satellite imagery and machine learning to predict poverty” (Science, August 19, 2016) .

Anyone may register here to receive connection details on this Zoom webinar (it’s free!). Please distribute this link to others who may be interested, especially colleagues and students.

The Computational Sustainability Virtual Seminar Series first talk is scheduled for

Tuesday, September 27, 2016
4:00 – 5:00  pm Eastern Time (1:00 – 2:00 pm Pacific Time)

Measuring progress towards sustainable development goals with machine learning
Stefano Ermon, Stanford University

Recent technological developments are creating new spatio-temporal data streams that contain a wealth of information relevant to sustainable development goals. Modern AI techniques have the potential to yield accurate, inexpensive, and highly scalable models to inform research and policy. As a first example, I will present a machine learning method we developed to predict and map poverty in developing countries. Our method can reliably predict economic well-being using only high-resolution satellite imagery. Because images are passively collected in every corner of the world, our method can provide timely and accurate measurements in a very scalable end economic way, and could revolutionize efforts towards global poverty eradication. As a second example, I will present some ongoing work on monitoring agricultural and food security outcomes from space.ermon-map-of-poverty-092016

BIO: Stefano Ermon is currently an Assistant Professor in the Department of Computer Science at Stanford University, where he is affiliated with the Artificial Intelligence Laboratory and the Woods Institute for the Environment. He completed his PhD in computer science at Cornell in 2015. His research interests include techniques for scalable and accurate inference in graphical models, statistical modeling of data, large-scale combinatorial optimization, and robust decision making under uncertainty, and is motivated by a range of applications, in particular ones in the emerging field of computational sustainability. Stefano has won several awards, including two Best Student Paper Awards, one Runner-Up Prize, and a McMullen Fellowship.

The Computational Sustainability Virtual Seminar Series will present talks by researchers and educators in Computational Sustainability, and is being sponsored by CompSustNet, with support from the National Science Foundation’s Expeditions in Computing program.

Best regards,

Carla Gomes and Doug Fisher

Watch this video to learn more about the Ermon Lab’s “Combining Satellite Imagery and Machine Learning to predict poverty”

Event listing on the CompSustNet web site





CompSustNet collaborator Dr. Andrew Mude awarded 2016 Norman Borlaug Award for Field Research and Application by World Food Prize Foundation

By Cornell University Professor Chris Barrett
Christopher B. Barrett is the Deputy Dean and Dean of Academic Affairs of the College of Business and the Stephen B. & Janice G. Ashley Professor of Applied Economics and Management and an International Professor of Agriculture in the Charles H. Dyson School of Applied Economics and Management, a Professor of Economics in the Department of Economics, and a Fellow in the David R. Atkinson Center for a Sustainable Future at Cornell University.

Dr. Andrew Mude, a research scientist at the International Livestock Research Institute in Nairobi, earned his Ph.D at Cornell University

I first met Andrew when he was a first year Ph.D. candidate in Economics and he dropped by my office to chat about his interests in development, particularly in Africa. We had many stimulating conversations and I wound up supervising his dissertation, which ranged from applied microeconomic theory applied to questions of microfinance and informal lending for education – a common practice in rural Kenya – to empirical work on the functioning of farmers cooperatives based on original survey data he collected in rural Kenya.  Toward the end of his doctoral studies, Andrew worked as my research assistant on a project using data from the Kenyan government’s early warning system in the pastoral drylands. As we studied those data we uncovered systematic patterns that led us to believe that one might be able to predict herd losses statistically with some precision.

L-R: Andrew Mude (ILRI), Chris Barrett (Cornell University), Brenda Wandera (ILRI), two Ethiopian herders, Birhanu Taddesse (ILRI).  Dr. Mude working with colleagues from IRLI and Cornell University to retrofit and secure GPS-enabled tracking collars for cattle in the Borena region of Ethiopia.
L-R: Andrew Mude (ILRI), Chris Barrett (Cornell University), Brenda Wandera (ILRI), two Ethiopian herders, Birhanu Taddesse (ILRI). Dr. Mude working with colleagues from IRLI and Cornell University to retrofit and secure GPS-enabled tracking collars for cattle in the Borena region of Ethiopia.

Another of my Ph.D. students, Sommarat Chantarat (Economics PhD 2009) then took up the challenge of designing an index-based livestock insurance (IBLI) contract in her dissertation, which won national and international awards. Dr. Chantarat, Dr. Mude – who moved to the Nairobi-based International Livestock Research Institute (ILRI) after completing his Cornell degree — and I worked closely with other partners to get IBLI designed and commercially piloted beginning in early 2010. We have had an extensive collaborative work agreement with Dr. Mude’s team at ILRI since 2008, which has supported Dr. Chantarat’s work as well as a more recent Ph.D., Nathan Jensen (AEM, 2014) whose dissertation evaluating the impacts and uptake of IBLI also won international recognition. This has been an especially enjoyable, fruitful, and impactful collaboration, which we are delighted continues quite actively today. The range of projects we have undertaken together has broadened over time, increasingly including computationally intensive work with Institute for Computational Sustainability  faculty and staff.

Andrew_playing_IBLI_Game_in_Kargi_D87A7A6606BF2This work to help address systemic risk among some of the poorest and most marginalized populations in the world is tremendously important and exciting. It is an enormous privilege to work with a collaborator as talented, committed and kind as Dr. Mude.

Click here to read the  press release by International Livestock Research Institute, all rights reserved, 2016

Working across borders is essential for birds, but also people

Opinion originally published in The Hill on August 26, 2016
By Amanda D. Rodewald
With political divisiveness so often headlining the news, how refreshing it is to celebrate a centennial that demonstrates the power of countries coming together. One hundred years ago, the United States and Great Britain (for Canada) came together for birds when they signed the Migratory Bird Treaty, a convention to protect migratory birds across international borders.
At the time, populations of many birds were plummeting due to poorly regulated hunting. The plume trade, for which an estimated 5 million birds — especially waterbirds like egrets and herons — were killed each year for feathers to adorn hats, eventually incited people to action. In response, the landmark treaty and subsequent act to enforce it (the Migratory Bird Treaty Act) protected more than 1,100 migratory bird species by making it illegal to pursue, hunt, take, capture, kill or sell live or dead birds, feathers, eggs and nests, except as permitted through hunting regulations for game birds.
What made this treaty particularly inspiring was that President Woodrow Wilson and King George V made the pledge amid the chaos of World War I. Soon after, the treaty was used as a model for similar agreements with Mexico (in 1936), Japan (1972) and Russia (1976). The Migratory Bird Treaties and subsequent international collaborations to conserve birds show how, when taken together, these global and hemispheric actions are far more than the sum of their parts. Collectively, these efforts are paving a path forward to protect birds and the ecosystems on which both birds and people depend.

International efforts are necessary to conserve migratory birds because birds don’t recognize geopolitical borders. Over the course of a year, songbirds, like the magnolia warbler, may spend 80 days breeding in the boreal forests of the northern U.S. and Canada, 30 days at resting and refueling sites during migration, and over 200 days overwintering in Latin American countries like Mexico, Belize, Guatemala and Honduras.

Looking back at the last 100 years, it’s heartening to see that where we acted together, we had success. The plume trade was virtually halted, and populations of herons, egrets and ibises rebounded. Waterfowl, too, have benefited tremendously from multinational habitat restoration and careful hunting management, in part guided by the North American Waterfowl Management Plan and transformative legislation like the North American Wetlands Conservation Act that leveraged billions in funding for restoration and conservation of over 30 million acres in the U.S., Canada and Mexico. Hunters and sportspeople were instrumental in this success, as they became the key drivers of conservation. Our investments have paid off: Populations of waterfowl and other waterbirds have increased, and the wetlands protected along the way now keep our drinking water clean and reduce flood risk.

But we need to do more. Unlike a century ago, when hunting decimated bird populations, today’s threats are more insidious. Birds are often collateral damage when habitat is lost and exotic species invade, and they are further endangered by collision with buildings and other structures, contamination, and even seemingly innocuous choices like letting our cats roam freely outdoors. These pervasive threats span geopolitical borders and, consequently, are best addressed through coordinated international action. Indeed, the recent “The State of North America’s Birds 2016” report indicates that without conservation action, over one-third of all North American bird species are at risk of extinction.

As we approach the next 100 years of conservation, we must remember that bird conservation is not only about birds; it’s about people, as well. We derive so many benefits from healthy bird populations, including pollination, seed dispersal, insect control and other ecosystem services. Birds also help us to understand the world around us and connect us with nature. And birds are a critical economic resource as well. Activities like hunting and birdwatching contribute billions of dollars to the U.S. economy alone. In fact, the North American Bird Conservation Initiative was originally created by the governments of Canada, the U.S. and Mexico in 1999 to recognize birds as an international “natural economic resource.”

The bottom line is that habitats healthy for birds are also healthy for people. We must work with diverse partners and stakeholders to identify conservation approaches, like sustainable forestry, that meet the needs of local communities and conserve birds and other species. Looking ahead, I believe that an integrated approach, where social and ecological needs are both accommodated, will be the hallmark of the next century of bird conservation.

Rodewald is the Garvin Professor of ornithology and director of conservation science at the Cornell Lab of Ornithology, faculty in the Department of Natural Resources at Cornell University, and faculty fellow at Cornell University’s Atkinson Center for a Sustainable Future. Views expressed in her column are hers alone and do not represent those of these institutions.

The views expressed by contributors are their own and not the views of The Hill.

The Institute for Computational Sustainability supports Code Red club

By Christianne White and Karina Burbank

The Institute for Computational Sustainability was proud to support Ithaca High School’s Code Red club’s  2016 build and competition. The Code Red Robotics Team 639 is part of the FIRST (For Inspiration and Recognition of Science and Technology) groups founded by Dean Kamen. In 2016 more than 75,000 students in more than 3,000 teams in 10 countries participated in 74 regional competitions that led to 1 championship competition. ICS supported the local Ithaca High School team Code Red during the six-week build session when students and their mentors design, build and test a robot, and then participate in successive rounds of competition.   Institute for Computational Sustainability especially admires the teamwork and creative problem-solving fostered by Code Red, which will prepare students well for further study in computer science, engineering, and design.

At Code Red students get help and advice from Ithaca High tech teachers but also mentors from the world outside the school. Karina Burbank, Code Red’s public relations officer writes, “Many of these mentors are engineers or other business professionals in the community, who give their time to help students during build season. While Code Red is a student-led team and we always make sure our students are involved and make major decisions, our mentors are vital to our success. They advise and guide students, helping every team member learn as much as possible, each day. This past season, we were lucky to have 16 community mentors.”

Freshman member Julian Perry works on the 2016 robot Predator’s frame, under the guidance of team mentors. This robot was designed, built and assembled during the 6-week build season.

Luvelle Brown, superintendent of Ithaca City School District commented on the relevancy of the club,” Code Red Robotics has done the best job of any program in the district that I can think of at truly integrating technology, at relationship-building between students and adults, and relevancy, making the curriculum come alive in a way that engages and makes the work important to young people.”

Anyone is welcome to join, and students can participate in a number of ways, so it’s not just for tech-trained nerds or kids who are super smart in math and physics.  Many kids participate all four years of high school even though the time commitment during the six weeks of ‘build’ commits them to at least 31 hours each week over and above their regular classes and homework.

Seniors Pooja Reddy and Kenzo Uchigasaki work together on the 2016 robot frame, even as electronic components are being installed at the front. The team's electronics subteam designs and assembles all of the robot's electronics. Other subteams handle programming, media, CAD, and machining.
Seniors Pooja Reddy and Kenzo Uchigasaki work together on the 2016 robot frame, even as electronic components are being installed at the front. The team’s electronics subteam designs and assembles all of the robot’s electronics. Other subteams handle programming, media, CAD, and machining.

From January to March, Code Red designs and builds their robot to meet the challenge sent out by FIRST, and develops their strategy to use during competitions.    Karina explains, “This challenge is different every year, but it’s always some sort of game that has two alliances of three robots competing against each other. This year’s game was called FIRST Stronghold, and our robot had to cross a series of defenses like a drawbridge, portcullis, and other similar things, as well as shoot balls into a goal, and pull itself up on a five-foot high bar.”

For some students, it’s all about winning, and Code Red did very well during 2016.  Karina let us know that, ”This year, we won the Fingerlakes Regional competition, as well as the Engineering Inspiration Award at Fingerlakes Regional (which celebrates outreach and education about STEM and robotics in a community), both of which qualified us for World Championships. At World Championships, we placed 6th in our division, and got to our division’s semi-finals as an alliance leader. (There are so many teams at champs that they are divided into several, simultaneously playing divisions.) This puts us in the top 2% of teams world-wide this year.”

The FIRST competitions are designed so teams must form alliances and cooperate with other teams in order to win. A few years ago Code Red was instrumental in helping nearby Trumansburg form a team, and this year Trumansburg’s team brought a great robot to competition. In April Team 639:Code Red Robotics posted on their Facebook page,” We’re going to elimination matches!! We won our last qualification match, placing us in 6th place in our division! This means that tomorrow, we will be able to select our own alliance to compete with in the finals. We think that this is the first time in Code Red history that we’ve been alliance captain in Champs elimination rounds, though we’ve participated as part of an alliance before.’  The Ithaca Journal reported that Trumansburg’s team co-founder Kevin Griswold was grateful to Ithaca’s Code Red team, “In our first year when we first formed the team, the whole group of us attended Code Red’s first meeting just to see how they ran things and to use them in a leadership role and as a role model for how we wanted to project ourselves,” Griswold said. “It’s only grown from there. Last year, we collaborated on a few practice runs to compete and practice against each other and did that again this year at the end of build season just to have another team to practice with.”

Code Red is not just about building robots. Karina helped us understand the wider scope of the club’s activities.  “Outside of the build season, we do a lot of community service and demos. This past year, there were 56 students on our team and we estimate that in total, including the individual community service we ask our team members to do, we did about 2,360 hours of community service. We also frequently do demonstrations and presentations about our robot and engineering, especially to young kids at the Sciencenter, 4-H, and local schools. Community service and outreach is a huge part of our team, and we always try to be as active in our community as possible, and help spread knowledge about Science, Technology, Engineering, and Math (STEM) to our community.”  Casey Dill, a Code Red and IHS  alumnus says, ”It’s a team that’s really trying to help the community,  teaching people and getting the younger generation excited about robotics and engineering before they ever get to the high school. Younger kids get a light in their eyes when they see what’s going on.”

Senior Kieran Loehr and junior Abigail Lee talk about the 2014 robot (now used as a demo robot) at the YMCA Healthy Kids Day. At the demo events team members show kids how the robot moves and plays games, and encourage them to come up to touch the robot and ask questions.
Senior Kieran Loehr and junior Abigail Lee talk about the 2014 robot (now used as a demo robot) at the YMCA Healthy Kids Day. At the demo events team members show kids how the robot moves and plays games, and encourage them to come up to touch the robot and ask questions.

Professor Gomes likes to support projects that are both competitive and cooperative, like Code Red.  She is recognized as a leader in the computational sustainability field, having received a second 10-million dollar grant from the National Science Foundation to develop the field of Computational Sustainability, so her own projects are both cutting-edge and collaborative, bringing together multiple scientists and institutions from a wide range of fields.

To see Team 639 in action watch this documentary produced by Ithaca College students.


IHS junior Code Red member Abigail Lee and mentor Jim Bedore volunteer with Code Red at the Ithaca Sciencenter outdoor playground. Code Red frequently volunteers as a team around Ithaca, to clean up parks, work at the Ithaca Children's Gardens, and more.
IHS junior Code Red member Abigail Lee and mentor Jim Bedore volunteer with Code Red at the Ithaca Sciencenter outdoor playground. Code Red frequently volunteers as a team around Ithaca, to clean up parks, work at the Ithaca Children’s Gardens, and more.



Promoting Computational Sustainability to the Public

This is a post by Selina Chen. See her bio at the bottom of the post.

When I attended the 4th International Conference on Computational Sustainability at Cornell University, I was surprised at the large number of projects that came with some involvement in incentivizing users — whether by making the technology more accessible for the average user or coming up with ways to get people interested in using the product.  Before the conference, I had thought research consisted of the studying and making of things, and the issue of actually “selling” the product to public would be one which would be foisted off to the companies who decided to use the research.  Doubtless, my view was clouded by the fact that that I’d been raised by two biochemists, whose jobs consisted of developing and testing new drugs, which would then be marketed and sold by another company or department in that company.  Yet, as I learned at the conference, this cycle of create-and-pass-off was not so in the CompSust research community. Oftentimes, it became the job of researchers to figure out how to properly integrate sustainability into public life by paying attention to behavioral science, as well as technology.

In her blog post last week, Zimei Bian talked about how the new mobile app, Pokémon GO, is sweeping the world and how games, and the concept of fun in particular, can be used to engage the public in sustainability efforts.  Also mentioned was Cornell’s eBird project, a web and mobile application that uses crowdsourcing to document the presence and absence of various bird species around the world.  Through this app, birdwatchers can submit data on birds found in their region and can even take a look at hotspots in other regions.  The inexperienced birdwatcher or the casual user who just likes to use the app for kicks can also use Merlin, another mobile app developed by the Cornell Lab of Ornithology, in conjunction with eBird to help identify birds.

Video: A promotion for Merlin Bird ID App (1 minute 54 sec) — Credit to Cornell Lab of Ornithology

But the Cornell Lab of Ornithology didn’t stop there!  Taking advantage of public’s love of games and competition, the lab, in close collaboration with the Cornell Institute for Computational Sustainability,  also developed a treasure-hunting app to go along with eBird called Avicaching, described  in the talk by Yexiang Xue of Cornell University.   The game combines eBird and geocaching to encourage users to search for birds in underrepresented locations by letting you earn a variable number of points for each location visited.  Equipped with a leaderboard that updates in real-time, the game encourages users to visit places with the most ‘points’, helping to reduce eBird’s sampling bias and collect a more accurate distribution of the bird population for scientists to use.  Currently, the game is in the process of development, having only been deployed in two New York counties, but with the initial success seen by the game, it may potentially be expanded into other regions.

Though they were quite memorable for how they built on top of and supported each other, eBird, Merlin, and Avicaching were not the only wonderful projects aimed at selling computational sustainability to public that stuck out to me at the conference.

Another great example of a project that had some focus on public perception included a talk given by David Shmoys of Cornell University on the rebalancing problem found in bike sharing.  Keeping bike racks in balance amidst fluctuating demand is key; a full bike rack will prevent users from stashing their bikes when they’re done and an empty one will deter potential users.  Therefore, bike racks must be managed and optimized to provide an appropriate level of stock at any given time during the day.

A photo of a Citibike station in New York -- credit to Wikimedia user Jim.henderson
A photo of a Citibike station in New York — credit to Wikimedia user Jim.henderson

Similarly, in taxi systems, there is also a demand problem and the issue of user perception.  In his talk on Smart Cities, Bo An of Nanyang Technological University described the peak time dilemma faced by many taxi users in cities: not being able to get a taxi. Because most taxis are priced by distance and because traffic is very slow during peak times, many taxi drivers will simply not work during those hours! The drivers do not believe that driving during peak times is cost effective.  Just like any other user, taxi drivers will attempt to game the system to their advantage, finding the best times to work to maximize their profit.  In order to “sell” the idea of working during peak periods, an incentive like raised fare prices must be brought to the table.  Further detail on this pricing and scheduling scheme can be found here. Its also important that we study possible rebound effects that might result from making “dysfunctional systems more tolerable“.

In a field like computational sustainability, which aims to touch all lives, human and otherwise, now and in the future, by building a better tomorrow, it is especially important that the public is “sold” on what we’re creating, lest the phrase “Everybody’s innovating, nobody’s integrating” becomes applied to us.  Fortunately, as seen by the conference, many researchers are already a step ahead and are proactive.

The projects mentioned here were only some of my favorite ones from the conference that touched on the idea of “selling” sustainability to the public. A complete list of talks hosted at the conference can be found here.

Selina Chen is a Computer Science  undergraduate at Vanderbilt University, with a love for sci-fi novels, superheroes, and art.  Currently, she’s having fun exploring the various ways art can be used to make data beautiful and engaging for the public.The opinions expressed herein are Selina’s and not necessarily those of Cornell University. You can reach Selina at Selina.Chen@vanderbilt.edu.


Computational Sustainability Community Blog

Skip to toolbar