Set the parent pivot to have a pick mode:
pivotParent->SetPickMode(PICK_COLLIDER);
Don't use the pick filter, since this will abort when the pivot is encountered:
//auto pick_info = world->Pick(pivot->GetPosition(true), target_pos, 0.25f, true, PickFilter);
auto pick_info = world->Pick(pivot->GetPosition(true), target_pos, 0.25f, true);
This is how the system is meant to work.