Opportunistic Computing with Lobster: Lessons Learned from Scaling up to 25k Non-Dedicated Cores

We previously described Lobster, a workflow management tool for exploiting volatile opportunistic computing resources for computation in HEP. We will discuss the various challenges that have been encountered while scaling up the simultaneous CPU core utilization and the software improvements required to overcome these challenges.