Office of Naval Research: N000141010933

ONR MURI Kick-off Meeting on

Knowledge Representation, Reasoning and Learning for Understanding Scenes and Events

[UCLA Campus, Public Affairs, Room 2355]

Day 1 – Friday, Oct 1

Session 1: Overview


Opening remarks on ONR and MURI
Behzad Kamghar-Parsi


Overview of the MURI project
Song-Chun Zhu


Coffee break

Session 2: Technology: object, scene and event

Jitendra Malik chairs this session


Object modeling and recognition
Deva Ramanan (20’) Detecting people & objects; where are we and what next?
Pietro Perona (20’) Desiderata for the next generation of class models
Jitendra Malik (20’) Top-down/bottom-up information flows in detection and segmentation


Scene categorization and recognition
Aude Oliva (30’) Scene and Object Recognition by Humans
Antonio Torralba (20’) Contextual Scene representation and Recognition
Daphne Koller (20’) Holistic spatial models of outdoor and indoor scenes


Lunch break
[There are many lunch places on campus with a variety of food]


Action and event understanding
Josh Tenenbaum (30’) Modeling how humans understand actions and events
Pietro Perona (20’) Our humble beginnings with flies and mice


Synergy between object, scene, and event and discussion
Pietro Perona (20’) Modeling mutual context between objects and human poses for activity recognition (representing the work of Fei-Fei)
Benjamin Yao (20’) Learning object relations and functions from actions and inferring goals and intents


Coffee break

Session 3: Theory: representation, learning and inference

Pietro Perona chairs this session


Issues on knowledge representation and grammatical models
David Mumford


Knowledge representation, acquisition, reasoning and generalization
— AI and cognitive perspectives

Josh Tenenbaum (20’) Principles of knowledge representation and generalization in human conceptual systems: Probabilistic programs and Hierarchical Bayes
Aude Oliva (25’) Representation of visual concepts in long term memory
Daphne Koller (20’) Beyond IID: Transfer Learning and Curriculum Learning
Judea Pearl (30’) Event Analysis through Counterfactuals


Dinner at Napa Valley Grille on 1100 Glendon Ave.


Interested people can attend the UCLA Jazz Concert at Schoenberg Hall

Day 2 – Saturday, Oct 2

Session 3: continued


Learning in the data laden and data scarcity regimes and compositional, grammatical models
George Papandreou (20’) Hierarchical Probabilistic Models for Visual Processing (representing Alan Yuille who is out of the country)
Tianfu Wu (20’) Top-down and bottom-up inference in hierarchical models
Stuart Geman (30’) Challenges to Hierarchical Modeling: Are they Insurmountable?


Theoretical foundation for modeling, learning and inference
Yingnian Wu (30’) From Information Scaling to regimes of Statistical Models
Song-Chun Zhu (30’) Towards a unified foundation of knowledge representation


Coffee break

Session 4: Dataset, Benchmarks, and Plans

Song-Chun Zhu chairs this session


Dataset, annotation, and benchmarking
Jitendra Malik (lead)
Antonio Torralba


Discussions on Milestones and deliverables
Pietro Perona (lead)
all other members will discuss


[We ordered 40 lunch boxes, will be served in the meeting room]


The Public Affairs building is shown in red. It is just cross the street from the UCLA guest house.

For people driving to UCLA: you need to enter from the intersection at Hilgard Ave and Westholme Ave (see the information sign (i) at the bottom of the small circle below). This is the information booth. You can get a parking permit only at this booth, NOT at the parking lot gate. The nearest parking lot is Parking Structure No.3 (see P3 at the top of the small circle).

A map of the UCLA campus can be downloaded here [

‘url’ attribute must set be for an existing page