There are a few projects already in existence that might be more convenient, than an rpi5 like fairphones, and I think the grapheneos team is looking to develop something too.

Mobile telephony support just comes as a module, so that’s actually the easy part.

The harder parts are to make the whole thing consume a low enough amount of power that you can keep it running from a non-monster-sized battery, and I suspect that an RPi 5 board isn’t very good for that (hobbyist development boards tend to not have been designed to avoid wasting power, even when the underlying microcontroller/processors is actually decent at it), and integrating an OS with support for a touch interface, especially if you want to avoid Android.

I mean, it’s not too hard to make a brick sized dumb phone and even have it be a mobile phone powered by AA batteries, but if you want a mobile smartphone, it gets more complex.

Unless you have the time and skills to take up the challenge you would probably be better of getting something like a Volla phone with Ubuntu Touch or a Pine phone.