Einstein wrote about replacing continuum physics with a discontinuum physics in 1935, 1936, 1952, and 1954. But he was not sure about how to proceed with the discontinuum physics.

In 1935, Einstein wrote the following to Paul Langevin.^1,2
“In any case one does not have the right today to maintain that the foundations must consist in a field theory in the sense of Maxwell. The other possibility, however, leads in my opinion to a renunciation of the time-space continuum and to a purely algebraic physics. Logically this is quite possible (the system is described by a number of integers; “time” is only a possible viewpoint [Gesichtspinkt]. From which the other “observables” can be considered --- an observable logically coordinated to all the others. Such a theory doesn’t have to be based upon the probability concept. For the present, however, instinct rebels against such a theory (Einstein to Paul Langevin, 3 October 1935, as translated in Stachel 1986, 379-80).”^1,2

In 1936, Einstein expressed the following in print.^1,2
“It has been suggested that, in view of the molecular structure of all events in the small, the introduction of a space-time continuum may be considered as contrary to nature. Perhaps the success of Heisenberg’s method points to a purely algebraical method of description of nature, to the elimination of continuous functions from physics. Then, however, we must also give up, on principle, the utilization of the space-time continuum. It is not inconceivable that human ingenuity will some day find methods that will make it possible to proceed along this path. Meanwhile, however, this project resembles the attempt to breath in an airless space (“Physics and Reality,” cited from Einstein 1954, 319, translation modified).^1,2

In 1952, Einstein wrote the following to Herbert Kondo.^1,2
“In the present-day physics there is manifested a kind of battle between the particle-concept and the field-concept for leadership, which will probably not be decided for a long time. It is even doubtful if one of the two rivals finally will be able to maintain itself as a fundamental concept (Einstein to Herbert Kondo, 11 August 1953, as translated in Stachel 1986, 380).”^1,2

On 10 August 1954, Einstein wrote the following to Michele Besso.^1,2
“I consider it entirely possible that physics cannot be based upon the field concept, that is on continuous structures. Then nothing will remain of my whole castle in the air including the theory of gravitation, but also nothing of the rest of contemporary physics (Einstein to Besso, 10 August 1954, as translated in [Stachel 1986, 380).”^1,2

On 24 August 1954, Einstein wrote the following to H. S. Joachim. ^1,2
“The alternative continuum-discontinuum seems to me to be a real alternative; i.e., there is here no compromise. By discontinuum theory I understand one in which there are no differential quotients. In such a theory space and time cannot occur, but only numbers and number-fields and rules for the formation of such on the basis of algebraic rules with exclusion of limiting processes. Which way will prove itself, only success can teach us.

Physics up to now is naturally in its essence a continuum physics, in spite of the use of the material point, which looks like a discontinuous conceptual element, and has no more right to existence in field description. Its strength lies in the fact that it posits parts which exist quasi-independently, beside one another. Upon this rest the fact that there are reasonable laws, that is rules which can be formulated and tested for the individual parts. Its weakness lies in the fact that it has not been possible up to now to see how that atomistic aspect including quantum relations can result as a consequence. On the other hand dimensionality (as four-dimensionally) lies at the foundation of the theory.”^1,2

An algebraic theory of physics id affected with just the inverted advantages and weaknesses, aside from the fact that on one has been able to propose a possible logical scheme for such a theory. It would be especially difficult to derive something like a spatio-temporal quasi-order from such a scheme. I cannot imagine how the axiomatic framework of such a physics would appear, and I don’t like it when one talks about it in dark apostrophes [Anredungen]. But I hold it entirely possible that the development will lead there; for it seems that the state of any finite spatially limited may be fully characterized by a finite number of numbers. This speaks against the continuum with its infinitely many degrees of freedom. The objection is not decisive only because one doesn’t know, in the contemporary state of mathematics, in what way the demand for freedom from singularity (in the continuum theory) limits the manifold of solutions (Einstein to H. S. Joachim, 24 August 1954, as translated in [Stachel 1986, 380).”^1,2

In October 1954, Einstein wrote to the following to David Bohm.^1,2
“I must confess that I was not able to find a way to explain the atomistic character of nature. My opinion is that if the objective description through the field as an elementary concept is not possible, then one has to find a possibility to avoid the continuum (together with space and time) altogether. But I have not the slightest idea what kind of elementary concepts could be used in such a theory (Einstein to David Bohm, 28 October 1954, as translated in [Stachel 1986, 380).”^1,2

References
¹J. Stachel, Einstein from ‘B’ to ‘Z’, Birkhauser, Boston, 2002, ISBN-13 978-0817641436.
²J. Stachel, Einstein from ’B’ to ’Z (Springer Science & Business Media, 2001), Vol. 9, pp. 150–152.