If we want to use computers as creative agents in the process of architectural design, we must first understand
how they can reason about architecture. I am taking a first small step towards the interpretation of architecture
by computers, trying to identify similarities and infer influences in the work of different architects, using
self organizing maps (SOM).
Project type
Pilot study || Research project with implementation
Steps
Implement a self organizing map algorithm.
Create a dataset to use with the algorithm. Identify architectural characteristics easily quantifiable
through drawings and add them to a database.
Conduct experiments running the algorithm, calibrating weights of different properties according to
importance.
Evaluate results and draw conclusions about the efficiency of the proposed methodology.
Contributions
Proposed a methodology for identifying similarities between architectural projects based on their
drawings.
Implemented a SOM, created dataset, and ran experiments.
Demonstrated that the use of self organizing maps is an efficient (or is not an efficient) method to
infer similarities in architectural projects.
Michael Behr
Symbol Grounding with Deep Neural Nets
If we want to understand human intelligence, we need to understand how
symbols can be grounded in perception. Stephen Larson's work on
self-organizing maps has shown us that we can give meaning to symbols
by using the statistical properties of our perceptual inputs to learn
representations of the world. Recent work on deep neural nets has
given us more expressive and efficient ways to learn representations
from statistical properties. By using deep neural nets, I will
replicate Larson's work on symbol grounding with more powerful
methods.
Project type
Research project with implementation.
Steps
I will construct a deep neural net to learn the statistical
regularities of a simple blocks world in an unsupervised manner.
I will extend my neural net to associate statistical regularities with
symbols in a supervised manner.
Contributions
Implemented a system to ground symbols in perception.
Compared self-organizing maps and deep neural nets as methods for
learning representations of perceptual data.
Demonstrated how deep neural nets can be applied to help understand
human intelligence.
Pedro Cattori
Chess Story Understanding
Chess computers have outgrown their human counterparts when it comes to the mechanics of the game. Yet, chess computers do not know what chess is. An easy way to see this is to try to learn from a chess program:
Human: "Deep Blue, why did you make that move?"
Deep Blue: "I have a strong instinctual urge to make that move."
Human: "How about the next move?"
Deep Blue: "Looking ahead 17 moves, the position is clearly winning for black."
At best, chess computers may come up with answers like these. But imagine if they could respond with the emotional and semantic understanding of human Grand Masters.
Human: "Deep Blue, why did you make that move?"
Deep Blue: "It reinforces my control of the center by unpinning my knight. It also allows me to recapture with my bishop, so I can avoid doubling my pawns."
Project type
Implementation: Genesis Extension.
Steps
To acquire semantic understanding of chess, I will equip the Genesis Story Understanding program with a chess-specific module.
(1) The first step is to enable the START system to ingest chess vocabulary by finding a suitable subset of English that richly describes chess and can also be used as Genesese.
(2) The second step is to provide genesis with a basic description of the real-world objects of chess, namely: the board (including the squares a1-h8 as objects), the pieces, and the timer.
(3) The third step is to implement a library of tactical and strategic terminology. Tactical terminology refers to concepts such as: skewering, forking, forcing a move, taking a piece, etc... Strategic terminology refers to concepts such as: king safety, center-control, castling, counterattacking, developing pieces, etc...
(4) The 4th step is to combine steps 1-3 in order to give Genesis a basic story understanding of chess games. The aim is to be able to ask Genesis simple questions about the course of a game at this point.
(5) Additional steps can be taken if time permits. This can include communicating with a chess engine via API for deeper analysis (such as sacrifice detection and trap detection). Another idea is to feed the output of Genesis back into chess engines who could make use of high level ideas more easily through a story than through a sequence of moves.
Contributions
Demonstrate the ease or difficulty for adding a highly-specific module to Genesis. In particular, it will be interesting to see how Genesis copes with an extremely structured story in the form of a chess game, but also one where pieces are the actors and the board is the stage.
Implement story understanding for chess computers. If a general API can be made, presumably this implementation could allow any chess engine to gain semantic understanding of chess and explain its move choice to humans.
Alexandros Charidis
The Role of Verbal Descriptions in Constructing Non-linguistic Representations of Spatial Compositions
Spatial compositions such as architectural compositions are primarily understood through
verbal descriptions, communicated orally or in written form, and visual descriptions,
communicated through drawings and physical models. The act of understanding a spatial
composition is most commonly known as reading. Then, the reading of a spatial
composition becomes plausible and comprehensible when its verbal description and its
visual description are aligned, that is, what you talk about, is what you show. I ask to
what extent, and under what conditions, a verbal description of a spatial composition
suffices to communicate and build a visual imagery, a non-linguistic representation, of
its structure. If one removes visual descriptions from a reading, what sorts of
constraints and heuristics in verbal descriptions enable one’s space-representing
apparatus to construct a unified account of the described composition?
Project type
Research Project with Implementation.
Steps
Preparation for an Experiment. I have made a set of spatial compositions giving emphasis in their structural aspects - their geometry and spatial relations - with little or no information about external to the structure information such as function, use, or material.
The Experiment. Subjects (A-subjects) will be asked to read the structure of a spatial composition, and build its verbal description. Different subjects (B-subjects) will be asked to read the description and construct a visual representation of the composition through drawing. A-subjects will be guided throughout their reading.
Analysis and Evaluation. Assuming that every reader builds a linguistic encoding of space that differs from the one another reader builds. I will analyze how these different encodings affect the non-linguistic conceptualization of a spatial composition by examining the existence, the extent of use, and sequence of space understanding concepts such as geometry, figures of speech (e.g. analogies and similarities). How much they overlap, and what sorts of errors they elucidate, what are their limits in language encoding of spatial concepts? A description might not need to include an excessive amount of detail to be useful and meaningful. So how much detail is actually needed?
Contributions
Developed a methodology for translating one’s visual perception of a spatial
composition into symbols, and from symbols translating back into visual imagery.
Designed and conducted an experiment to test the methodology.
Exposed the heuristics behind the intelligent reasoning involved when building a non-linguistic representation (visual imagery) of a spatial composition based purely on its verbal description.
Showed that constraint exposing story telling is the central foundation for thinking on how to communicate verbally the structure of a spatial composition.
Michaela Ennis
Story Understanding in Schizophrenia
Schizophrenia is thought to be an exclusively human disease. Understanding what types of cognitive errors cause the disorder is interesting not only for its own sake, but also because it may elucidate some of the unique mechanisms of human intelligence. Schizophrenia includes not only perceptual hallucinations, but also delusions. It is possible that the cause of such delusions is due to a problem in pattern matching. To investigate this, potential mechanisms through which pattern matching may be impaired can be investigated computationally. I propose using the Genesis system to examine this question.
Project type
Research proposal.
Steps
First, stories will be gathered from past studies that showed delusion in Schizophrenia. Example stories are necessary to feed into the system, and the outlandish conclusions that patients drew from these stories are necessary to confirm that any excessive pattern matching is reflective of actual symptoms.
Next, various mechanisms for issues in pattern matching will be investigated. Among those are a tendency to draw excessive inferences, a tendency to connect events that are not related, relaxed criteria for matching concepts, and some concepts that are themselves too vague. All of these have the potential to contribute to over-applying a concept, and each can break in various ways. These mechanisms can be investigated by reviewing literature in biology, psychology, and computer science.
Finally, although beyond the scope of this particular project, the mechanisms decided to be most likely can be implemented in the Genesis system and tested.
Contributions
Will examine the different levels of story processing at which Schizophrenic patients may form delusions.
Will propose a method to test potential mechanisms for a failure of pattern matching in Schizophrenia.
Christopher Grimm
What does Genesis Feel?
Feelings play an important role in human expression starting at a really young age and are often used to describe things we lack a sufficient
vocabulary to explain in detail. For example a crying toddler who has just had their favorite toy stolen might describe themself as feeling sad
due to an insufficient vocabulary or understanding of complex concepts such as injustice. Even adults use feelings to describe things beyond their
descriptive powers such as a young couple blaming irrational actions on love. Imagine if a machine were capable of expressing itself with feelings.
It would be capable of performing feats beyond its descriptive capacity. Machine learning could be performed by encouraging a machine to perform
actions it would view as fun, while avoiding actions that would scare it or make it feel sad. An intelligent airplane could describe itself as
worried to a human pilot instead of listing off a series of indiscernible problems. Genesis could suggest a book to a human reader based off what
it believes is exciting.
Project type
Research project with implementation.
Steps
Identify a series of potential feeling that Genesis might be capable of such as excitment, boredom, happiness, and sadness.
Analyze how different stories make humans feel and why these stories bring about those feelings.
Analyze patterns in Genesis's understanding of stories to determine how Genesis might feel excitment, boredom, happiness, etc.
Demonstrate that it is possible for Genesis to group stories based on how it feels about the stories.
Contributions
Will analyze why humans feel certain ways toward certain stories.
Will demonstrate that Genesis mirrors these feelings in its understanding of stories.
Will design a proof-of-concept program that sorts stories by how they make Genesis feel.
Beth Hadley
The Influence of Culture on Genesis Story Understanding
Culture defines many fundamental aspects of human behavior. In order to build robust intelligence systems that can interpret information about the world and make human-like inferences, it is imperative that the system incorporates cultural understanding.
Project type
Research project with Genesis implementation
Steps
Identify specific examples from literature where differences between high and low context cultures lead to different behavior in humans.
Write these examples as stories Genesis can understand and make inferences about.
Express the cultural differences as rules in a language Genesis can understand so that they may be used to interpret the stories written earlier.
Run the stories through Genesis, including which cultural rules to use in the interpretation of the story. Study the resulting output, and ensure that Genesis makes the correct inferences based on the differing cultural interpretation.
Analyze the above process to more deeply understand how computers can understand the influence of culture on human behavior. Conclude by making generalized recommendations regarding how to incorporate cultural awareness into future artificial intelligence systems.
Contributions
Identified specific examples of how culture impacts human behavior, thoughts, and actions.
Demonstrated the ability to provide rules based on cultural awareness to influence how an artificial intelligence system interprets information.
Facilitated the ability to include cultural awareness into artificial intelligence systems.
Abigail Klein
Genetic Graphs of Continuous Knowledge
Online classrooms are becoming increasingly popular with the creation of EdX, Coursera, and Khan Academy. In order for these systems to cater to each individual student, like a human teacher might, we first need to be able to model a student’s knowledge of the material.
Ira P. Goldstein proposed a model for procedural knowledge, or subjects based on discrete steps, in his The Genetic Epistemology of Rule Systems. He represents a procedure as a graph, where nodes are rules and links are relationships. For example, in a graph of arithmetic, one node might be learning that single-digit addition (1+1=2) and another node might be learning double-digit addition (10+10=20). The link between them might be to explain that we may explain that we may add the digits separately (1+1=2 and 0+0=0).
While this seems to be a reasonable representation of learning concepts whose steps are discrete, it might not transfer to a more continuous skill, such as learning to write, play a sport, or play a musical instrument. Skills such as these cannot be broken into discrete sub-steps, but rather have various sub-concepts which are improved simultaneously over time.
In this project, I will extend Goldstein’s model for procedural knowledge to model continuous knowledge, which will ultimately be useful in creating intelligent computer-based teachers. I will use a genetic graph of pole vaulting as an example to illustrate this model.
Project type
Research project with implementation
Steps
Define continuous knowledge.
Identify the sub-concepts in pole vaulting.
Attempt to fit a genetic graph of pole vaulting to Goldstein’s model in order to determine the limitations.
Adapt Goldstein’s model to accommodate these limitations.
Try to represent continuous nature of sub-concepts as regions.
Try to represent continuous nature of sub-concepts as discrete steps, as in motion planning.
Formalize the changes to Goldstein’s model and generalize to other subjects with continuous knowledge.
Contributions
Will adapt Goldstein’s genetic graphs of procedural knowledge to continuous knowledge.
Will create a genetic graph of pole vaulting.
Nick Locascio
Automated Programming in a Lightbot World
The role of many software engineers is to take a function specification and implement a function that fits the specification and solves the given problem. The general quality of such an implementation relies on the following questions (in order): 1) does it work? 2) is it readable? 3) is it concise? 4) is it modular and reusable? When it comes down to it, many, if not most, of the tasks assigned to a software engineer require little creativity and ingenuity. This is not to fault software engineers but to highlight the valuable constraints that specifications provide. Specifications are and should be written such that little to no inference is needed to determine what the desired action is. A good spec means that a software engineer just has translate natural language into instructions in a programming language. Once translated, the engineer tries to refactor the code to fit the quality marks. Such a task, when defined in this way, seems ripe for automation.
The automation of software engineering will lead to a second explosion of software solutions. Both the cost and time of building software will plummet dramatically, and the accessibility of software development will be unprecedented. Software development will be as easy as speaking, as it will consist of giving instructions to the automated software engineer. Automation could also create a safer more secure world if and when the automated programming agent exceeds human ability.
The vision for this project specifically is to take a step toward the grand vision of automated programming by creating an Automated Programming agent capable of writing programs to solve problems in the world of Lightbot.
Project type
Research project with implementation.
Steps
The following steps outline a general path toward achieving the vision. This specific project aims to tackle the first few in a highly constrained world, but nonetheless the steps are as follows.
Design and implement a search-based routine that finds a program that solves the given problem.
Design and implement a compression scheme that minimizes program length by placing commonly used sequences of operations into functions.
Expand compression scheme to allow functions to be called from other functions. This expanded routine is our refactoring agent.
Modify refactoring agent to consider multiple problems and their solutions at a time, thereby reducing total codebase size.
Design and implement story-telling routine explaining what the program does in natural language.
Modify compression scheme to tell the simplest story, not to create the shortest program.
Design an extended language in which functions can take parameters.
Design a refactoring scheme capable of creating functions with parameters.
Design and implement story-understanding routine capable of recreating a solution from its own story.
Design and implement story-understanding routine capable of creating a solution from a human crafted natural language specification.
Bootstrap automated programmer program to be written by earlier incarnations of itself.
Contributions
Will implement search-based program generator to solve Lightbot worlds.
Will propose a general metric for evaluating the quality of Lightbot code based on brevity, modularity, and reusability.
Will implement an automated code-refactoring system that aims to maximize the quality of the software by placing commonly used sequences of operations into functions.
Will develop a proof-of-concept program that solves Lightbot programming puzzles in an optimal way by generating a solution then optimizing the code structure.
Will generalize system so that it can optimize multiple Lightbot puzzles at once with the goal of maximizing the quality of the entire codebase.
Will extend the automatic programmer to explain its solution as a story in goal oriented fashion.
Will implement a code-refactoring routine that optimizes for story simplicity rather than code structure.
Will implement basic story-understanding routine to find a solution from story.
Ziqin (Shaun) Rong and Manuel Cabral
Fooling Larson
Evolutionary algorithms have proven effective in fooling deep neural network systems, rendering the latter imperfect to be the full solution of computer vision. Then what is the ultimate visual recognition system for computers? This project tests if Larson's Intrisic Representation System is more robust and can handle genetic algorithms deliberately designed to fool them.
Project type
Research project with implementation
Steps
Read through Larson's graduate thesis and related literatures about evolution algorithms.
Build up Larson's Intrisic Representation System
Train the Intrisic Representation System with digital number images (0 9, represented by a 7-dimensional vector
Build up evolutionary input feeding system upon the trained Intrisic Representation System, trying to see if it can be fooled with meanless input vector.
Contributions
Trained a Intrinsic Representation System to recognize digital numbers.
Tested Intrinsic Representation System's performance against algorithms designed to deliberately fool it.
Sandoval Olascoaga, Carlos
Making the Identity of a Place through Synthetic Visual Representations
It is possible to understand how the identity of a place changes through time and cause by conjoining visual data of the place with verbal descriptions of its spatial aspects.
Project type
Research project with implementation.
Steps
I have collected historical images, social and geographical data of certain regions of New York City.
I am building a platform that can display the historical images and their corresponding data sets and that allows users to construct causal relationships between the spatial aspects of the place.
Subjects will be asked to use the platform and create causal relationships of the places using the synthetic representations and the different historic images that the platform shows.
The different graphs constructed by the users will be classified based on the graphs created through unsupervised methods to create a visual memory of the places.
Contributions
Implemented an interactive platform for causal place making.
Showed what type of stories do places embody through visual representations of their spatial characteristics and through verbal descriptions of their social, economical and geographical aspects.
Proposed a method of collecting and representing the causal relationships that build a perception of a place.
Eann Tuan
Using Genesis to Understand Stories from the Bible
Humans learn, understand, and develop through story telling and story analysis.Genesis, an artifically intelligent program that reads and analyzes stories, tests a suite of constraint exposing representations. It uses human generated concepts and rules to make common sense conclusions. Currently, Genesis models concepts from stories written by Shakespeare, such as Macbeth, but my vision for this project challenges Genesis to reflect on its reading of stories from the Bible, searching for concepts and demonstrate basic story understanding.
Project type
Research project with implementation.
Steps
Select stories from the Bible with common themes, such as revenge and murder, and convert them into Geneses, the subset of English understood by Gensis. This will involve creating a file containing all the sentences needed to understand a simple story, including prediction, explanation, and presumption rules. Then, I will feed this file into Genesis to build an elaboration graph that will display common sense rules connecting explicit and inferred elements of the story.
Contributions
I will propose a new domain that demonstrates Genesis' ability to develop basic story understanding and use story concepts to make common sense conclusions based on stories from the Bible.
Nikolaos Vlavianos
Plethora: Perceiving Design through Embodied Cognition
Given the infinite world of design possibilities, Plethora project attempts to rule and frame design outputs through a dual process of creating and testing. If architects understand the way that non-architects perceive designs and vice versa, then designs will become a launching pad for both agents to communicate with less ambiguity. The word design in Plethora project denotes a representation of the meaning of a communicative act between architects and non-architects. In this case, a physical model is committed to a compositional account of meaning by which a vast space of ideas can be composed from a relatively small set of elementary ideas. The purpose of communication between a design output and an architect or a non-architect is tightly connected with a process of sharing and materializing those ideas in the physical world. An experiment will run in two separate iterations for architects and non-architects.
Project type
Research project with implementation.
Steps
Subjects will be asked to draw a 2D representation of a given physical model. Five people will design five representations, and they will vote two that are closer to the original physical model.
Based on the given representations, another group will be asked to pick one physical model from a sequence of eleven.
At this point, I will notice a local deviation of the selected model from the original.
Then, the subjects will repeat step one and two.
Each group of subjects will be asked to tell a story on how they perceived design,
Finally, they will highlight four locations of a physical model that were attracted their interest.
This process will be repeated twice testing both a group of architects and non-architects from a random pull of MIT students.
I will draw conclusions from this data about whether architects and non-architects perceive designs in a different way, or whether there is an overlap of concepts that share similar attributes.
Contributions
Collected and analyzed data tested on whether physical models or 2D representations can be perceived in a similar way.
Both architects and non-architects explained design as a set of rules and concepts that were applied by the designer.
Architects shared an almost uniform vocabulary to describe design, while non-architects referred to those concepts with a variety of words.
Both Architects and non-architects demonstrated that design perception is not related to any sort of training, but a prior experience helped them to narrate the story of a design or to invent a new story from scratch.
Through this experiment a set of concepts and words from different disciplines emerged giving rise to a library of ideas for future designs.
