Photo Sleuth
A human-AI collaborative platform for researching historical photos
Identifying people in historical photographs is important for preserving material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. We introduce Civil War Photo Sleuth (CWPS), a free, public website where users can identify unknown photos from the American Civil War era (1861-65). The platform leverages the complementary strengths of facial recognition and crowdsourced human expertise.
Carbon Neutrality
Eco-Feedback Interventions for Encouraging Green Vehicular Choices
We explore effective ways to communicate carbon footprint information i.e., does CO2 information in grams/pounds make more sense than relatable terms such as the number of trees saved or gallons of gasoline burnt? We tested how different CO2-based interventions can nudge users to make eco-friendly decisions in the context of ride hailing and car rentals.
This work was done as part of my internship at Toyota Research Institute.
Blog post coming soon
Civil War Twin
ETHICAL DESIGN challenges in facial recognition
Facial recognition systems pose numerous ethical challenges around privacy, racial and gender bias, and accuracy, yet little guidance is available for designers and developers. We explore solutions to these challenges in a three-phase design process to create Civil War Twin (CWT), an educational web-based application where users can discover their lookalikes from the American Civil War era (1861-65) while learning more about facial recognition and history. Through this design process, we operationalize a framework for AI literacy, consult with scholars of history, gender, and race, and evaluate CWT in feedback sessions with diverse prospective users.
Blog post coming soon
DoubleCheck
community-based assessability workflows for historical photo id verification
Misidentified historical photos can have significant negative consequences, including lost economic value, incorrect historical records, and the spread of misinformation that can lead to perpetuating conspiracy theories. We introduce DoubleCheck, a quality assessment framework based on the concepts of information provenance and stewardship, for assessing the quality of historical photo IDs on Civil War Photo Sleuth (CWPS).
Blog post coming soon
Second Opinion
CROWDSOURCED FEEDBACK pipeline for addressing ai last-mile challenges
AI-based discovery tools often present a human expert with a shortlist of high-confidence candidates from which the expert must select the correct match(es) while avoiding false positives; we term this as the “last-mile problem of AI.” Inspired by cognitive science theories of similarity, we propose Second Opinion, an online tool that employs a novel crowdsourcing workflow “seed-gather-analyze” to assist experts in solving the last-mile problem of person identification using facial recognition tools.
Blog post coming soon
Painted Past
a human-ai collaborative workflow for discovering 19th century painted studio backdrops
In historical photo research, the presence of painted backdrops have the potential to help triangulate subjects, photographers, locations, and events surrounding certain photographs. Yet, research processes around these backdrops are poorly documented, with no known tools to aid in the task. We introduce Painted Past, a human-AI collaborative system that incorporates computer vision techniques and novel user interactions for discovering and organizing photos sharing identical painted backdrops from the American Civil War era.
Blog post coming soon
Photo Steward
deliberative human-ai decision making workflows for validating historical photo identifications
The task of pinpointing identities in historical photos has significant importances but comes with inherent difficulties. Facial recognition, though helpful for assistance, has its limitations, often suggesting false positives that can affect user decisions. We introduce Photo Steward, an information stewardship architecture for validating historical photo identities with careful deliberation.
Blog post coming soon
VSCO
investigating user micro-communities on Vsco
Platforms such as Facebook, Twitter, Instagram, etc. are designed to be socially translucent with features reflecting real-world social interaction and networks, which essentially become a basis for how micro communities are formed on these platforms. We investigated micro communities on VSCO, a content-sharing platform geared towards artists and photographers, that is devoid of typical social media interactions while being opaque about the underlying social network. Without these features, we hypothesized that the user-generated content is responsible for how micro communities are formed on the platform.
This work was done as part of my internship at VSCO.
Blog post coming soon
SleuthTalk
private collaboration workspace with structured feedback and intelligent shortlists for identifying historical photos
Photo misidentifications, both modern and historical, can have significant repercussions. We introduce SleuthTalk, a private workspace, where users can collaboratively investigate historical photo identities using structured feedback and intelligently curated shortlists.
Blog post coming soon
Vikram Mohanty 