By "everything beautiful and lofty" the Underground Man means those things that are ideal or at least highly valued. In the context of his monologue, he is addressing people whose concept of the"beautiful and lofty" would include reason, order, structure, and maybe a few classically beautiful things. There is an obvious contrast between the Underground Man's lowly way of life, and the idyllic theories of life proposed by Chernyshevsky and his ilk.