The price of a barrier option depends on the shape of the entire implied volatility surface which is a high-dimensional dynamic object. Barrier options are hence exposed to non-trivial volatility risk. We extract the key risk factors of implied volatility surface fluctuations by means of a semiparametric factor model. Based on the factors we define a practical hedging procedure within a local volatility framework. The hedging performance is evaluated using DAX index options.