Inverse Reinforcement Learning with Suboptimal Experts

Reinforcement Learning is defined as: Given 1) measurements of an agent’s behavior over time in a variety of circumstances, 2) if needed, measurements of the sensory inputs of that agent; 3) if available, a model of the environment, determine the reward function being optimized. The problem definition does not talk about how well behaved the experts are, and if they know the exact model of the environment when they are taking actions. Therefore a general solution to this problem also should not include any assumptions as to whether the expert trajectories are optimal or not. However, existing works in this field either explicitly assume the expert trajectories are all optimal, or their algorithms tend to work poorly for sub-optimal expert trajectories. We propose a new algorithm which is much more resilient to sub-optimal trajectories.

Variance Reduction for Kacsmarz methods

Kacsmarz methods are very popular optimization methods but they are prone to non existence of optimal solution and noise in the data. My research focused on utilizing Variance Reduction algorithms to make Kacsmarz methods more robust.


CoCo is a visual analytics tool that enables users to compare two sets of temporal sequence data. It combines automated statistical tests with user-guidance to enable insights, hypothesis generation, and much more. Users see (1) statistics about their dataset, (2) event-level statistics, and (3) a menu of metrics. CoCo displays significance tests in a unified form for measures such as prevalence and duration of gaps.

Interaction Conflicts

Having several systems working autonomously in an environment can lead into conflict situations which should be dealt with in some way. We study these situations and conflict management methods.


Flipping is a local and efficient operation to construct the convex hull in an incremental fashion. However, it is known that the traditional flip algorithm is not able to compute the convex hull when applied to a polyhedron in R3. Our novel Flip-Flop algorithm is a variant of the flip algorithm. It overcomes the deficiency of the traditional one to always compute the convex hull of a given star-shaped polyhedron with provable correctness. Applying this to construct convex hull of a point set in R3, we develop ffHull, a flip algorithm that allows nonrestrictive insertion of many vertices before any flipping of edges. This is unlike the well-known incremental fashion of strictly alternating between inserting a single vertex and flipping. The new approach is not only simpler and more efficient for CPU implementation but also maps well to the massively parallel nature of the modern GPU. As shown in our experiments, ffHull running on the CPU is as fast as the best-known convex hull implementation, qHull. As for the GPU, ffHull also outperforms all known prior work. From this, we further obtain the first known solution to computing the 2D regular triangulation on the GPU.