浮点减法结果不正确

时间:2016-01-26 09:11:47

标签: c# sql sql-server floating-point

的数据集

   id   qty CheckIn CheckOut
   5    10      1     0
   5    10      0     1
   5    1.6     1     0
   5    0.4     0     1
   5    0.4     0     1
   5    0.4     0     1

我正在尝试选择id 5的剩余数量。

查询:

Select 
    (select SUM(qty) from tbl where id = 5 and CheckIn = 1) -
    (select SUM(qty) from tbl where id = 5 and CheckOut = 1)

预期输出: 4.0

原始输出: 3.9999999

即使是单个查询也会返回完美的值:

select SUM(qty) from tbl where id = 5 and CheckIn = 1  -- returns 11.6
select SUM(qty) from tbl where id = 5 and CheckOut = 1 -- returns 11.2

但减法给出了3.9999999

您可以使用以下查询生成数据:

以下是生成数据集的查询

CREATE TABLE [dbo].[tbl](
    [id] [int] NOT NULL,
    [qty] [float] NULL,
    [CheckIn] [bit] NULL,
    [CheckOut] [bit] NULL
) ON [PRIMARY]

GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 10, 1, 0)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 10, 0, 1)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 1.6, 1, 0)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 0.4, 0, 1)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 0.4, 0, 1)
GO
INSERT [dbo].[tbl] ([id], [qty], [CheckIn], [CheckOut]) VALUES (5, 0.4, 0, 1)
GO

2 个答案:

答案 0 :(得分:0)

SELECT
    CAST(SUM(CASE WHEN CheckIn = 1 THEN qty END) AS MONEY)
    -
    CAST(SUM(CASE WHEN CheckOut = 1 THEN qty END) AS MONEY)
FROM tbl
WHERE id = 5

答案 1 :(得分:0)

问题通过将数据类型更改为十进制来解决