First, make sure you have used Nuget task with restore command to restore the dependency packages in the pipeline. If the problem persists, you can check if there are any conflicts between the packages you have in source control and the package you are trying to restore.
In addition, you can try to disable the local cache in the build definition. The local cache may only be partially filled.
Here is a ticket with similar issue you can refer to.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…