Can the fastener nut/collar protrude on the back side? If yes, use potting in the core with a thru hole and nut/collar on the back side. If no, then use an insert, which itelf may need potting in the core.
In either case, to get the panel pull thru/out capability you will need test data for the particular configuration.