Felipe Toledo (MSc)

Felipe Toledo (MSc)

Ph.D student at the University of Virginia. Interested in AI and Deep Neural Network validation

Publications

T4PC: Training Deep Neural Networks for Property Conformance

Felipe Toledo, Trey Woodlief, Sebastian Elbaum, and Matthew B. Dwyer

The increasing integration of Deep Neural Networks (DNNs) into safety critical systems, such as Autonomous Vehicles (AVs), where failures can lead to significant consequences, has fostered the development of many Verification and Validation (V&V) techniques. However, these techniques are applied mainly after the DNN training process is complete. This delayed…
Read more

Content: [Pre-print] [Ieee-xplore] [Artifact]

Closing the Gap Between Sensor Inputs and Driving Properties: A Scene Graph Generator for CARLA

Trey Woodlief, Felipe Toledo, Sebastian Elbaum, and Matthew B. Dwyer

The software engineering community has increasingly taken up the task of assuring safety in autonomous driving systems by applying software engineering principles to create techniques to develop, validate, and verify these systems. However, developing and analyzing these techniques requires extensive sensor datasets and execution infrastructure with the relevant features and…
Read more

Content: [Pre-print] [Paper] [Artifact]

Monitoring Safety Properties for Autonomous Driving Systems with Vision-Language Models

Felipe Toledo, Sebastian Elbaum, Divya Gopinath, Ramneet Kaur, Ravi Mangal, Corina S. Pasareanu, Anirban Roy, and Susmit Jha

With the increased adoption of autonomous vehicles comes the need to ensure they reliably follow safe driving properties. Formally specifying and monitoring such properties is challenging because of the semantic mismatch between the high-level properties (e.g., assertions on spatial relationships between the ego vehicle and other entities in a road…
Read more

Content: [Pre-print] [Artifact]

Scene Flow Specifications: Encoding and Monitoring Rich Temporal Safety Properties of Autonomous Systems

Trey Woodlief, Felipe Toledo, Sebastian Elbaum, and Matthew B. Dwyer

To ensure the safety of autonomous systems, it is imperative for them to abide by their safety properties. The specification of such safety properties is challenging because of the gap between the input sensor space (e.g., pixels, point clouds) and the semantic space over which safety properties are specified (e.g….
Read more

Content: [Pre-print] [Paper] [Artifact]

RBT4DNN: Requirements-based Testing of Neural Networks

Nusrat Jahan Mozumder, Felipe Toledo, Swaroopa Dola, and Matthew B. Dwyer

Deep neural network (DNN) testing is crucial for the reliability and safety of critical systems, where failures can have severe consequences. Although various techniques have been developed to create robustness test suites, requirements-based testing for DNNs remains largely unexplored - yet such tests are recognized as an essential component of…
Read more

Content: [Pre-print] [Artifact]

The SGSM framework: Enabling the specification and monitor synthesis of safe driving properties through scene graphs

Trey Woodlief, Felipe Toledo, Sebastian Elbaum, and Matthew B. Dwyer

As autonomous vehicles (AVs) become mainstream, assuring that they operate in accordance with safe driving properties becomes paramount. The ability to specify and monitor driving properties is at the center of such assurance. Yet, the mismatch between the semantic space over which typical driving properties are asserted (e.g., vehicles, pedestrians)…
Read more

Content: [Pre-print] [Paper] [Artifact]

Specifying and Monitoring Safe Driving Properties with Scene Graphs

Felipe Toledo, Trey Woodlief, Sebastian Elbaum, and Matthew B. Dwyer

With the proliferation of autonomous vehicles (AVs) comes the need to ensure they abide to safe driving properties. Specifying and monitoring such properties, however, is challenging because of the mismatch between the semantic space over which typical driving properties are asserted (e.g., vehicles, pedestrians, intersections) and the sensed inputs of…
Read more

Content: [Pre-print] [Paper] [Artifact] [Video]

S3C Spatial Semantic Scene Coverage for Autonomous Vehicles

Trey Woodlief, Felipe Toledo, Sebastian Elbaum, and Matthew B. Dwyer

Autonomous vehicles (AVs) must be able to operate in a wide range of scenarios including those in the long tail distribution that include rare but safety-critical events. The collection of sensor input and expected output datasets from such scenarios is crucial for the development and testing of such systems. Yet,…
Read more

Content: [Pre-print] [Paper] [Artifact]

Deeper Notions of Correctness in Image-Based DNNs: Lifting Properties from Pixel to Entities

Felipe Toledo, David Shriver, Sebastian Elbaum, and Matthew B. Dwyer

Deep Neural Networks (DNNs) that process images are being widely used for many safety-critical tasks, from autonomous vehicles to medical diagnosis. Currently, DNN correctness properties are defined at the pixel level over the entire input. Such properties are useful to expose system failures related to sensor noise or adversarial attacks,…
Read more

Content: [Pre-print] [Paper] [Video]

Distribution Models for Falsification and Verification of DNNs

Felipe Toledo, David Shriver, Sebastian Elbaum, and Matthew B. Dwyer

DNN validation and verification approaches that are input distribution agnostic waste effort on irrelevant inputs and report false property violations. Drawing on the large body of work on model-based validation and verification of traditional systems, we introduce the first approach that leverages environmental models to focus DNN falsification and verification…
Read more

Content: [Paper] [Appendix] [Artifact] [Video]