Uncategorized – Paul Reverdy

UMD Robotics Seminar

November 16, 2020 PaulLeave a comment

I was honored to give the Lockheed Martin Robotics Seminar at the University of Maryland last month. I talked about motivation dynamics, the framework I am developing to use dynamical systems tools for autonomous task management.

Much of the material I discussed can be found in various publications from my group, especially [1] and [2].

Giving talks over Zoom is an art I’m still learning. Showing videos poses a particular challenge because they’re often transmitted with a lag. For those who want to take a closer look at my videos from that talk, I post them here for reference.

A point robot using motivation dynamics to navigate in a sphere world, repeatedly patrolling two goal locations (red diamonds) while avoiding obstacles (black circles)

State trajectories for the two obstacles simulation. From top to bottom, the panels show navigation functions (normalized distance to goals), motivation state, and value state.

Simulation of the identical controller as in the two obstacle case, but now with one moving obstacle. The controller has no knowledge of the obstacle’s intent, just a perfect sensing of the obstacle’s current location.

State trajectories for the one moving obstacle simulation. From top to bottom, the panels show navigation functions (normalized distance to goals), motivation state, and value state.

Video for our recently-accepted T-RO paper on Motivation Dynamics [2].

[1]

P. B. Reverdy and D. E. Koditschek, “A dynamical system for prioritizing and coordinating motivations,” SIAM Journal on Applied Dynamical Systems, vol. 17, iss. 2, p. 1683–1715, 2018.
[Bibtex]

@article{PBR-DEK:18,
Author = {Reverdy, Paul B and Koditschek, Daniel E},
Date-Added = {2018-02-05 06:22:16 +0000},
Date-Modified = {2018-10-10 19:27:06 -0700},
Journal = {{SIAM Journal on Applied Dynamical Systems}},
Number = {2},
Pages = {1683--1715},
Pdf = {https://arxiv.org/abs/1703.01662},
Title = {A dynamical system for prioritizing and coordinating motivations},
Url = {https://epubs.siam.org/doi/pdf/10.1137/17M111972X},
Volume = {17},
Year = {2018},
Bdsk-Url-1 = {https://arxiv.org/abs/1703.01662}}

[2]

P. Reverdy, V. Vasilopoulos, and D. E. Koditschek, “Motivation dynamics for autonomous composition of navigation tasks,” IEEE Transactions on Robotics, 2020.
[Bibtex]

@article{PBR-VV-DEK:20,
Author = {P. Reverdy and V. Vasilopoulos and Daniel E. Koditschek},
Date-Added = {2018-10-31 22:05:02 -0700},
Date-Modified = {2020-10-28 22:09:18 -0700},
Journal = {{IEEE} {T}ransactions on {R}obotics},
Note = {In press},
Pdf = {http://www.paulreverdy.com/wp-content/papercite-data/pdf/PBR-VV-DEK-20.pdf},
Title = {Motivation dynamics for autonomous composition of navigation tasks},
Year = {2020}}

New Amazon role

November 15, 2020November 15, 2020 PaulLeave a comment

Since July 2020 I have been on leave from the University and working on navigation for Amazon Scout, a last-mile delivery robot. For more information, see this story at amazon.science.

All posts here reflect my personal opinions and do not reflect official Amazon positions.

Teaching award

November 15, 2020 PaulLeave a comment

It’s been a busy few months, but I am honored to have been selected for the 2020 Awards of Excellence at the Student Interface by AME’s graduate students. Thanks to all who voted.

SICK LIDAR Challenge

October 1, 2019 PaulLeave a comment

A group of my students was one of the fifteen teams selected to take part in the SICK LIDAR challenge https://s.sick.com/us-en-TiM10k-2019. They will receive a TIM 781 lidar sensor to develop their project to measure surface roughness in desert environments.

Congratulations!

CDC papers

October 1, 2019 PaulLeave a comment

We have two new papers accepted to this year’s Conference on Decision and Control:

[1] Extends the results of our previous paper [2] to account for individual tasks encoded as limit cycles.

[3] Shows how the decision-making mechanism at the heart of the motivation dynamics system [2] embeds an unfolded pitchfork bifurcation.

Hope to see you in Nice!

[1]

C. Thompson and P. Reverdy, “Drive-based motivation for coordination of limit cycle behaviors,” in IEEE 58th Annual Conference on Decision and Control (CDC), 2019, p. 244–249.
[Bibtex]

@inproceedings{CT-PR:19,
Author = {C. Thompson and P. {Reverdy}},
Booktitle = {{IEEE} 58th {A}nnual {C}onference on {D}ecision and {C}ontrol ({CDC})},
Date-Added = {2019-09-12 09:52:31 -0700},
Date-Modified = {2020-01-06 14:50:32 -0700},
Pages = {244--249},
Pdf = {http://www.paulreverdy.com/wp-content/papercite-data/pdf/CT-PR-19.pdf},
Title = {Drive-Based Motivation for Coordination of Limit Cycle Behaviors},
Year = {2019}}

[2]

P. B. Reverdy and D. E. Koditschek, “A dynamical system for prioritizing and coordinating motivations,” SIAM Journal on Applied Dynamical Systems, vol. 17, iss. 2, p. 1683–1715, 2018.
[Bibtex]

@article{PBR-DEK:18,
Author = {Reverdy, Paul B and Koditschek, Daniel E},
Date-Added = {2018-02-05 06:22:16 +0000},
Date-Modified = {2018-10-10 19:27:06 -0700},
Journal = {{SIAM Journal on Applied Dynamical Systems}},
Number = {2},
Pages = {1683--1715},
Pdf = {https://arxiv.org/abs/1703.01662},
Title = {A dynamical system for prioritizing and coordinating motivations},
Url = {https://epubs.siam.org/doi/pdf/10.1137/17M111972X},
Volume = {17},
Year = {2018},
Bdsk-Url-1 = {https://arxiv.org/abs/1703.01662}}

[3]

P. B. Reverdy, “Two paths to finding the pitchfork bifurcation in motivation dynamics,” in IEEE 58th Annual Conference on Decision and Control (CDC), 2019, p. 8030–8035.
[Bibtex]

@inproceedings{PBR:19d,
Author = {P. B. {Reverdy}},
Booktitle = {{IEEE} 58th {A}nnual {C}onference on {D}ecision and {C}ontrol ({CDC})},
Date-Added = {2019-09-12 09:56:54 -0700},
Date-Modified = {2020-01-06 14:49:38 -0700},
Pages = {8030--8035},
Pdf = {http://www.paulreverdy.com/wp-content/papercite-data/pdf/PBR-19d.pdf},
Title = {Two paths to finding the pitchfork bifurcation in motivation dynamics},
Year = {2019}}

SIAM Dynamical Systems talk video

June 21, 2019June 21, 2019 PaulLeave a comment

My session at the SIAM Snowbird conference was recorded; you can see video of my talk on the web here.

As I mentioned at the beginning of the talk, I was fighting a cold at the time, so please forgive the state of my voice!

Thanks again to the session organizers.

SoCal Robotics Symposium

May 2, 2019May 2, 2019 PaulLeave a comment

Last weekend we were at Caltech for the 2019 SoCal Robotics Symposium. It was a great, small conference with really interesting ideas from academia, the Jet Propulsion Laboratory, and industry.

I presented our work on motivation dynamics [1]. The extended abstract is available here [2] and the summary poster slide is below. Thanks to the organizers for a well-organized and stimulating day.

soCalRobotics2019 Download

[1]

P. B. Reverdy and D. E. Koditschek, “A dynamical system for prioritizing and coordinating motivations,” SIAM Journal on Applied Dynamical Systems, vol. 17, iss. 2, p. 1683–1715, 2018.
[Bibtex]

@article{PBR-DEK:18,
Author = {Reverdy, Paul B and Koditschek, Daniel E},
Date-Added = {2018-02-05 06:22:16 +0000},
Date-Modified = {2018-10-10 19:27:06 -0700},
Journal = {{SIAM Journal on Applied Dynamical Systems}},
Number = {2},
Pages = {1683--1715},
Pdf = {https://arxiv.org/abs/1703.01662},
Title = {A dynamical system for prioritizing and coordinating motivations},
Url = {https://epubs.siam.org/doi/pdf/10.1137/17M111972X},
Volume = {17},
Year = {2018},
Bdsk-Url-1 = {https://arxiv.org/abs/1703.01662}}

[2]

P. B. Reverdy, “Value-based decision making for human-machine robot control,” in Southern California robotics symposium, 2019.
[Bibtex]

@inproceedings{PBR-19c,
Author = {Reverdy, Paul B},
Booktitle = {Southern {C}alifornia Robotics Symposium},
Date-Added = {2019-04-27 14:01:02 -0700},
Date-Modified = {2019-04-27 14:03:00 -0700},
Pdf = {http://www.paulreverdy.com/wp-content/papercite-data/pdf/PBR-19c.pdf},
Title = {Value-based decision making for human-machine robot control},
Year = {2019}}

UCL algorithm corrections

April 23, 2019May 1, 2019 PaulLeave a comment

There was a subtle but small error in the proofs published in [1]. We have corrected the error in a new appendix G added to the arXiv version of the paper, available at https://arxiv.org/abs/1307.6134v4. These corrections also apply to other papers which built off of the results in [1], including [2], and [3].

The error arose from our application of concentration inequalities, sometimes known as tail bounds. In the originally-published proofs, we condition on the number $\smash{n_i^t}$ of times that the algorithm has selected arm $i$ up to time $t$ . Since the arm selection policy depends on the rewards accrued, $\smash{n_i^t}$ and the rewards are dependent random variables. In the correction, we build upon an alternative concentration inequality that accounts for this dependence and show that proofs of all the performance bounds follow a similar pattern with slight modification to the decision heuristic.

[1]

P. B. Reverdy, V. Srivastava, and N. E. Leonard, “Modeling human decision making in generalized Gaussian multiarmed bandits,” Proceedings of the IEEE, vol. 102, iss. 4, p. 544–571, 2014.
[Bibtex]

@article{PBR-VS-NEL:14,
Author = {Reverdy, Paul B and Srivastava, Vaibhav and Leonard, Naomi Ehrich},
Date-Added = {2018-02-05 06:22:16 +0000},
Date-Modified = {2018-02-08 21:11:54 +0000},
Journal = {Proceedings of the {IEEE}},
Number = {4},
Pages = {544--571},
Pdf = {http://www.paulreverdy.com/wp-content/papercite-data/pdf/PBR-VS-NEL-14.pdf},
Publisher = {IEEE},
Title = {Modeling human decision making in generalized {G}aussian multiarmed bandits},
Volume = {102},
Year = {2014}}

[2]

P. Reverdy, V. Srivastava, and N. E. Leonard, “Satisficing in multi-armed bandit problems,” IEEE Transactions on Automatic Control, vol. 62, iss. 8, p. 3788–3803, 2017.
[Bibtex]

@article{PR-VS-NEL:17,
Author = {Reverdy, Paul and Srivastava, Vaibhav and Leonard, Naomi Ehrich},
Date-Added = {2018-02-05 06:22:16 +0000},
Date-Modified = {2018-02-08 21:34:38 +0000},
Doi = {10.1109/TAC.2016.2644380},
Journal = {{IEEE} {T}ransactions on {A}utomatic {C}ontrol},
Number = {8},
Pages = {3788--3803},
Publisher = {IEEE},
Title = {Satisficing in multi-armed bandit problems},
Volume = {62},
Year = {2017},
Bdsk-Url-1 = {https://doi.org/10.1109/TAC.2016.2644380}}

[3]

P. Reverdy and N. E. Leonard, “Parameter estimation in softmax decision-making models with linear objective functions,” IEEE Transactions on Automation Science and Engineering, vol. 13, iss. 1, p. 54–67, 2016.
[Bibtex]

@article{PR-NEL:16,
Author = {Reverdy, Paul and Leonard, Naomi Ehrich},
Date-Added = {2018-02-05 06:22:16 +0000},
Date-Modified = {2018-02-08 21:37:50 +0000},
Doi = {10.1109/TASE.2015.2499244},
Journal = {{IEEE} {T}ransactions on {A}utomation {S}cience and {E}ngineering},
Number = {1},
Pages = {54--67},
Pdf = {http://www.paulreverdy.com/wp-content/papercite-data/pdf/PR-NEL-16.pdf},
Publisher = {IEEE},
Title = {Parameter estimation in softmax decision-making models with linear objective functions},
Volume = {13},
Year = {2016},
Bdsk-Url-1 = {https://doi.org/10.1109/TASE.2015.2499244}}

SIAM Snowbird

April 13, 2019May 3, 2019 PaulLeave a comment

With the end of the semester fast approaching, we’re getting close to conference season! My student Craig Thompson and I will be speaking at SIAM’s Dynamical Systems conference in Snowbird next month.

My talk is part of a special session on nonlinear decision-making dynamics on Sunday and Craig’s talk is on Monday afternoon. Hope to see you there!

Students presenting at IMECE

September 21, 2018September 20, 2018 PaulLeave a comment

The undergraduates who worked in the lab last summer will be presenting two posters at the ASME IMECE conference based on their summer work. If you’re attending IMECE in Pittsburgh, please stop by on November 11!

Brendan Bogar will present “Investigating a Framework for Visualizing Reinforcement Learning Algorithms via Quadrupedal Robotic Simulation”.

David Chan, Mel Nguyen, Oshadha Gunasekara, and Randall Kliman will present “An object-oriented framework for fast development and testing of mobile robot control algorithms”. Abstracts are available on the IMECE website.