I do research combining complementary Artificial Intelligence techniques (from Machine Learning to Knowledge Representation). My major area of interest is Natural Language Processing as a means for data ingestion to and enrichment of knowledge graphs.
Selene Báez Santamaría
Computational Lexicology & Terminology Lab of Prof. Dr. Piek Vossen
Vrije Universiteit Amsterdam
De Boelelaan 1105
1081 HV Amsterdam
PhD candidate • March 2020 - present
The Vossen-Spinoza-project Understanding Language By Machines has funded a follow-up project on sub-project 5 “Make Robots talk” (Leolani): “MAKE ROBOTS TALK AND THINK”. In this Ph.D., I will study trust as a relationship between social agents in a multimodal world that involves multifaceted skills and complex contexts. This work aims to create and evaluate a computational model of trust, from the robot perspective towards trusting humans in collaborative tasks.
M.Sc. Degree in Artificial Intelligence (Cum Laude) • 2015 - 2017
Specialized on Intelligent Systems Design. Interested on Machine Learning, Deep Learning, Data Mining, Computer Vision, Multi-agent Systems, and NLP
Master thesis • February 2017 - June 2017
My thesis consisted on mining data from smart card users of the public transportation system in Beijing. The project includes supervised and unsupervised learning techniques. On the one hand, a classifier is created to identify public transit commuters via ensemble models. On the second hand, a convolutional autoencoder is used to engineer features that represent the travellers behaviours. These features are further clustered to identify seven common travelling patterns.
B.Sc. Degree in Cognitive Systems: Computational Intelligence & Design (Graduated with Distinction) • 2011 - 2014
Cognitive Systems is a multi-disciplinary program involving Computer Science, Linguistics, Philosophy, and Psychology.
AI Researcher • June 2019 - February 2020
Data Scientist • October 2017 - June 2019
myTomorrows provides patients with unmet medical needs, and their physicians, information about treatment options worldwide and facilitate access to medicines in development. My role focused on processing medical unstructured language to generate structured data in the form of a semantic knowledge graph.
University Research Fellow • September 2017 - August 2019
Supervised by prof. dr. Piek Vossen, I worked on the Spinoza project Understanding Language By Machines. This project focuses on enabling a Pepper robot (Leolani) to learn from language and context. Using Speech/Object recognition, NLP and Knowledge Representation techniques, it tackles problems of artificial cognition such as provenance, theory of mind, relevance and permanence.
Watson Demo Intern • July 2016 - January 2017
Create simple but powerful cognitive applications using the Watson Developer Cloud for backend services and Nao robots as interface. Perform end-to-end application development combining cognitive tools related to natural language, vision and speech.
I am a determined and goal-oriented person with strong analytical and critical thinking. I favor collaborative workplaces that promote self-growth through challenging projects.
The goal of this project is to predict the opponent’s configuration in a RoboCup SSL environment. For simplicity, a Markov model assumption is made such that the predicted formation of the opponent team only depends on its current formation. The field is divided into a grid and a robot state per player is created with information about its position and its velocity. To gather a more general sense of what the opposing team is doing, the state also incorporates the team’s average position (centroid). All possible state transitions are stored in a hash table that requires minimum storage space. The table is populated with transition probabilities that are learned by reading vision packages and counting the state transitions regardless of the specific robot player. Therefore, the computation during the game is reduced to interpreting a given vision package to assign each player to a state, and looking for the most likely state it will transition to. The confidence of the predicted team’s formation is the product of each individual player’s probability. The project is noteworthy in that it minimizes the time and space complexity requirements for opponent’s moves prediction.Robots, UBC
This project proposes a three dimensional representation for pattern recognition of traveling behavior. We carry out dimensionality reduction on this representation and compare supervised and unsupervised learning tasks for recognizing typical behaviors in users. First, ensemble models aid us on the task of binary classification of commuters (as labeled by self-reported survey data). Then, we characterize users by encoding their behaviors using an convolutional autoencoder, and performing clustering. The city of Beijing, China is posed as an use case for the project. With 75\% average accuracy, our results on binary classification are not competitive with other studies on the field. However, labeled data obtained via surveys is typically noisy, and thus a high classification rate does not necessarily benefit Transportation specialists. In contrast, we found 7 distinct user groups on the clustering task leading to a better understanding of the users' routines and needs. In conclusion, we believe state of the art techniques for unsupervised learning and local representations should be exploited in this domain. As such, the knowledge extracted can lead to better target policies and services in pro of improving the transportation network in large and complex cities.Machine Learning, VU
This paper presents a novel method for mining the individual travel behavior regularity of different public transport passengers through constructing travel behavior graph based model. The individual travel behavior graph is developed to represent spatial positions, time distributions, and travel routes and further forecasts the public transport passenger’s behavior choice. The proposed travel behavior graph is composed of macro-nodes, arcs, and transfer probability. Each macro-node corresponds to a travel association map and represents a travel behavior. A travel association map also contains its own nodes. The nodes of a travel association map are created when the processed travel chain data shows significant change. Thus, each node of three layers represents a significant change of spatial travel positions, travel time, and routes, respectively. Since a travel association map represents a travel behavior, the graph can be considered a sequence of travel behaviors. Through integrating travel association map and calculating the probabilities of the arcs, it is possible to construct a unique travel behavior graph for each passenger. The data used in this study are multimodal data matched by certain rules based on the data of public transport smart card transactions and network features. The case study results show that graph based method to model the individual travel behavior of public transport passengers is effective and feasible. Travel behavior graphs support customized public transport travel characteristics analysis and demand prediction.Graph Technologies, BJUT
Our state of mind is based on experiences and what other people tell us. This may result in conflicting information, uncertainty, and alternative facts. We present a robot that models relativity of knowledge and perception within social interaction following principles of the theory of mind. We utilized vision and speech capabilities on a Pepper robot to build an interaction model that stores the interpretations of perceptions and conversations in combination with provenance on its sources. The robot learns directly from what people tell it, possibly in relation to its perception. We demonstrate how the robot’s communication is driven by hunger to acquire more knowledge from and on people and objects, to resolve uncertainties and conflicts, and to share awareness of the perceived environment. Likewise, the robot can make reference to the world and its knowledge about the world and the encounters with people that yielded this knowledge.Robots, VU
This paper describes the system that team MYTOMORROWS-TU DELFT developed for the 2019 Social Media Mining for Health Applications (SMM4H) Shared Task 3, for the end-to-end normalization of ADR tweet mentions to their corresponding MEDDRA codes. For the first two steps, we reuse a state-of-theart approach, focusing our contribution on the final entity-linking step. For that we propose a simple Few-Shot learning approach, based on pre-trained word embeddings and data from the UMLS, combined with the provided training data. Our system (relaxed F1: 0.337- 0.345) outperforms the average (relaxed F1 0.2972) of the participants in this task, demonstrating the potential feasibility of few-shot learning in the context of medical text normalization.Health informatics, myTomorrows
We describe a model for a robot that learns about the world and her companions through natural language communication. The model supports open-domain learning, where the robot has a drive to learn about new concepts, new friends, and new properties of friends and concept instances. The robot tries to fill gaps, resolve uncertainties and resolve conflicts. The absorbed knowledge consists of everything people tell her, the situations and objects she perceives and whatever she finds on the web. The results of her interactions and perceptions are kept in an RDF triple store to enable reasoning over her knowledge and experiences. The robot uses a theory of mind to keep track of who said what, when and where. Accumulating knowledge results in complex states to which the robot needs to respond. In this paper, we look into two specific aspects of such complex knowledge states: 1) reflecting on the status of the knowledge acquired through a new notion of thoughts and 2) defining the context during which knowledge is acquired. Thoughts form the basis for drives on which the robot communicates. We capture episodic contexts to keep instances of objects apart across different locations, which results in differentiating the acquired knowledge over specific encounters. Both aspects make the communication more dynamic and result in more initiatives by the robot.Robots, VU
People and robots make mistakes and should therefore recognize and communicate about their“imperfectness” when they collaborate. In previous work [3, 2], we described a female robot model Leolani(L) that supports open-domain learning through natural language communication, having a drive to learn new information and build social relationships. The absorbed knowledge consists of everything people tell her and the situations and objects she perceives. For this demo, we focus on the symbolic representation of the resulting knowledge. We describe how L can query and reason over her knowledge and experiences as well as access the Semantic Web. As such, we envision L to become a semantic agent which people could naturally interact with.Robots, VU
This paper presents a model of contextual awareness implemented for a social communicative robot Leolani. Our model starts from the assumption that robots and humans need to establish a common ground about the world they share. This is not trivial as robots make many errors and start with little knowledge. As such, the context in which communication takes place can both help and complicate the interaction: if the context is interpreted correctly it helps in disambiguating the signals, but if it is interpreted wrongly it may distort interpretation. We defined the surrounding world as a spatial context, the communication as a discourse context and the interaction as a social context, which are all three interconnected and have an impact on each other. We model the result of the interpretations as symbolic knowledge (RDF) in a triple store to reason over the result, detect conflicts, uncertainty and gaps. We explain how our model tries to combine the contexts and the signal interpretation and we mention future directions of research to improve this complex process.Robots, VU
Language identification remains a challenge for short texts originating from social media. Moreover, domain specific terminology, which is frequent in the medical domain, may not change cross-linguistically, making language identification even more difficult. We conducted language identification on four datasets, two of them with general language, and two of them containing medical language. We evaluated the impact of two embedding representations and a set of linguistic features based on graphotactics. The proposed linguistic features reflect the graphotactics of the languages included in the test dataset. For classification, we implemented two algorithms: random forest and SVM. Our findings show that, when classifying general language, linguistic-based features perform close to the embedding representations of fastText and BERT. However, when classifying text with technical terms, the linguistic features outperform embedding representations. The combination of embeddings with linguistic features had a positive impact on the classification task under both settings. Therefore, our results suggest that these linguistic features could be applied for big and small datasets keeping the good performances in both general and medical languages. As future work, we want to test the linguistic features for a more significant set of languages.Health informatics, myTomorrows