2021年3月27日星期六

Why Decimal(20,10) returns better value of Pi than Decimal(38,10) in SQL Server

When executing the following query in SQL Server:

SELECT       CAST(22) AS decimal(38, 10) / CAST(7) AS decimal(38, 10) AS pie  

the output is: 3.142857.

But when I run the same query with lower precision of 20 I get more values after the decimal

SELECT      CAST(22) AS decimal(20, 10) / CAST(7) AS decimal(20, 10) AS pie  

The output is : 3.142857142857142857

Can anyone explain why this is so?

I have a similar situation in SQL Server where I need the accurate value up to 10 decimal places when I divide two decimal(38, 10) columns.

Problem being, these two columns are coming from a separate table, where they are defined as DECIMAL(38, 10).

https://stackoverflow.com/questions/66836033/why-decimal20-10-returns-better-value-of-pi-than-decimal38-10-in-sql-server March 28, 2021 at 04:59AM

没有评论:

发表评论