Continuous Authentication (CA) mechanisms involve managing sensitive data from users which may change over time. Both requirements (privacy and adapting to new users) lead to a tension in the amount and granularity of the data at stake. However, no previous work has addressed them together. This paper proposes a CA approach that leverages incremental Matrix Profile (MP) and Deep Learning using accelerometer data. Results show that MP is effective for CA purposes, leading to 99% of accuracy when a single user is authorized. Besides, the model can on-the-fly increase the set of authorized users up to 10 while offering similar accuracy rates. The amount of input data is also characterized -- the last 15 sec. of data in the user device require 0.4 MB of storage and lead to a CA accuracy of 97% even with 10 authorized users.