Make a graham cracker cake by combining 1/2 cup each of ground pecans and cake flour, 2 cups of graham cracker crumbs, 2 teaspoons of baking powder and 1/4 teaspoon of salt in one bowl. Beat together 1 cup of unsalted butter, 1 cup of light brown sugar, 3 egg yolks and 2 teaspoons of vanilla extract in another bowl. Whip 3 egg whites and 1/4 cup of sugar in a third bowl.
Stir the graham cracker crumb mixture and 1 cup of lukewarm milk in parts into the egg yolk mixture, then fold in the egg white mixture. Spoon the batter into two greased and parchment-lined round cake pans, and bake them at 350 degrees Fahrenheit for 25 minutes.
While the cakes cool, whisk in 2 tablespoons of melted sugar with 1/4 cup of softened butter and 1/2 cup of dark brown sugar. Bring the mixture to a boil, then add 1/2 cup of heavy whipping cream. Boil the sauce for one minute, remove it from the heat and beat in 3/4 cup of confectioners' sugar and 1 1/4 cups of butter. Spread the frosting between and over the cakes, and sprinkle them with 1/4 cup of ground pecans.